2023-10-07T07:42:24.998 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-07T07:42:24.998 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223 --verbose -- /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/orig.config.yaml 2023-10-07T07:42:25.044 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-07T07:42:25.130 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223 branch: wip-yuri-testing-2023-10-06-0949-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} email: yweinste@redhat.com first_in_suite: false job_id: '7416223' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri-testing-2023-10-06-0949-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-06-0949-quincy sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4726 sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-06-0949-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDvzTKOQgOGweamAbVUNfrC8pj/dSyjZ9V0tg0+C7IOj4n90Q1to5OmKeP+Jy9J1prx3MYhGj8ZNLJlE75U6p9w= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD/VRA/QSRFLcfZ0A2n4Zd6jOOh9njj/CIoNcl1+A3795u1aCYQmZKDV32dGZGcnYvugk/tG7M6tUZ9JXtW67cA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-07_00:40:05 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-07T07:42:25.131 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa; will attempt to use it 2023-10-07T07:42:25.132 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks 2023-10-07T07:42:25.132 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-07T07:42:25.133 INFO:teuthology.task.internal:Checking packages... 2023-10-07T07:42:25.156 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '08cac0187919fc3a247efdf445f652e2fb30a647' 2023-10-07T07:42:25.156 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-07T07:42:25.156 INFO:teuthology.packaging:ref: None 2023-10-07T07:42:25.156 INFO:teuthology.packaging:tag: None 2023-10-07T07:42:25.157 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-06-0949-quincy 2023-10-07T07:42:25.157 INFO:teuthology.packaging:sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T07:42:25.157 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-06-0949-quincy 2023-10-07T07:42:25.307 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1320.g08cac018 2023-10-07T07:42:25.307 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-07T07:42:25.313 INFO:teuthology.task.internal:no buildpackages task found 2023-10-07T07:42:25.313 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-07T07:42:25.371 INFO:teuthology.task.internal:Saving configuration 2023-10-07T07:42:25.388 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-07T07:42:25.394 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T07:42:25.419 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 07:37:17.738859', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDvzTKOQgOGweamAbVUNfrC8pj/dSyjZ9V0tg0+C7IOj4n90Q1to5OmKeP+Jy9J1prx3MYhGj8ZNLJlE75U6p9w='} 2023-10-07T07:42:25.444 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 07:37:17.736779', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD/VRA/QSRFLcfZ0A2n4Zd6jOOh9njj/CIoNcl1+A3795u1aCYQmZKDV32dGZGcnYvugk/tG7M6tUZ9JXtW67cA='} 2023-10-07T07:42:25.444 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-07T07:42:25.450 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-07T07:42:25.450 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-07T07:42:25.450 INFO:teuthology.run_tasks:Running task console_log... 2023-10-07T07:42:25.517 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4f98be1b80>, signals=[15]) 2023-10-07T07:42:25.518 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-07T07:42:25.523 INFO:teuthology.task.internal:Opening connections... 2023-10-07T07:42:25.523 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:42:25.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:42:25.595 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:42:25.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:42:25.665 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-07T07:42:25.673 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2023-10-07T07:42:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2023-10-07T07:42:25.698 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2023-10-07T07:42:25.760 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2023-10-07T07:42:25.761 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8" 2023-10-07T07:42:25.761 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2023-10-07T07:42:25.761 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2023-10-07T07:42:25.761 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8" 2023-10-07T07:42:25.761 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8" 2023-10-07T07:42:25.761 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-07T07:42:25.761 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2023-10-07T07:42:25.761 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-07T07:42:25.761 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2023-10-07T07:42:25.762 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-07T07:42:25.762 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-07T07:42:25.762 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-07T07:42:25.762 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2023-10-07T07:42:25.787 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2023-10-07T07:42:25.809 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2023-10-07T07:42:25.810 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2023-10-07T07:42:25.871 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2023-10-07T07:42:25.871 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8" 2023-10-07T07:42:25.871 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2023-10-07T07:42:25.871 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2023-10-07T07:42:25.871 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8" 2023-10-07T07:42:25.871 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2023-10-07T07:42:25.872 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-07T07:42:25.872 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-10-07T07:42:25.872 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-07T07:42:25.872 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2023-10-07T07:42:25.872 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-07T07:42:25.872 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-07T07:42:25.872 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-07T07:42:25.872 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-10-07T07:42:25.895 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-07T07:42:25.904 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-07T07:42:25.910 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-07T07:42:25.911 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2023-10-07T07:42:25.913 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2023-10-07T07:42:25.933 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-07T07:42:25.939 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-07T07:42:25.939 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2023-10-07T07:42:25.975 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2023-10-07T07:42:26.008 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-07T07:42:26.064 INFO:teuthology.run_tasks:Running task kernel... 2023-10-07T07:42:26.080 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-07T07:42:26.081 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-07T07:42:26.081 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-07T07:42:26.082 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-07T07:42:26.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T07:42:26.101 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2023-10-07T07:42:26.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T07:42:26.103 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2023-10-07T07:42:26.162 INFO:teuthology.orchestra.run.smithi172.stdout:4.18.0-489.el8.x86_64 2023-10-07T07:42:26.162 INFO:teuthology.task.kernel:Running kernel on smithi172: 4.18.0-489.el8.x86_64 2023-10-07T07:42:26.163 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum install -y kernel 2023-10-07T07:42:26.164 INFO:teuthology.orchestra.run.smithi007.stdout:4.18.0-489.el8.x86_64 2023-10-07T07:42:26.164 INFO:teuthology.task.kernel:Running kernel on smithi007: 4.18.0-489.el8.x86_64 2023-10-07T07:42:26.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2023-10-07T07:42:29.854 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 8 - AppStream 30 MB/s | 33 MB 00:01 2023-10-07T07:42:30.134 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 8 - AppStream 25 MB/s | 33 MB 00:01 2023-10-07T07:42:38.446 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 8 - BaseOS 19 MB/s | 48 MB 00:02 2023-10-07T07:42:38.892 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 8 - BaseOS 21 MB/s | 48 MB 00:02 2023-10-07T07:42:45.521 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 8 - Extras 120 kB/s | 18 kB 00:00 2023-10-07T07:42:46.313 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 8 - Extras common packages 11 kB/s | 6.9 kB 00:00 2023-10-07T07:42:46.458 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 8 - Extras 64 kB/s | 18 kB 00:00 2023-10-07T07:42:46.598 INFO:teuthology.orchestra.run.smithi172.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-10-07T07:42:48.457 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 16 MB 00:01 2023-10-07T07:42:52.307 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 355 kB/s | 24 kB 00:00 2023-10-07T07:42:55.806 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-07T07:42:55.806 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-07T07:42:56.075 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2023-10-07T07:42:56.075 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout: Package Architecture Version Repository Size 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2023-10-07T07:42:56.076 INFO:teuthology.orchestra.run.smithi172.stdout:Install 3 Packages 2023-10-07T07:42:56.077 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:42:56.077 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 88 M 2023-10-07T07:42:56.077 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 96 M 2023-10-07T07:42:56.077 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2023-10-07T07:42:56.852 INFO:teuthology.orchestra.run.smithi172.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 14 MB/s | 10 MB 00:00 2023-10-07T07:42:57.517 INFO:teuthology.orchestra.run.smithi172.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 25 MB/s | 35 MB 00:01 2023-10-07T07:42:58.208 INFO:teuthology.orchestra.run.smithi172.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 20 MB/s | 43 MB 00:02 2023-10-07T07:42:58.209 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2023-10-07T07:42:58.209 INFO:teuthology.orchestra.run.smithi172.stdout:Total 41 MB/s | 88 MB 00:02 2023-10-07T07:42:58.880 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2023-10-07T07:42:59.294 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2023-10-07T07:42:59.294 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2023-10-07T07:43:00.519 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2023-10-07T07:43:00.544 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2023-10-07T07:43:02.658 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 8 - Extras common packages 444 B/s | 6.9 kB 00:16 2023-10-07T07:43:03.288 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2023-10-07T07:43:03.802 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-07T07:43:04.126 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 2.7 kB/s | 3.5 kB 00:01 2023-10-07T07:43:05.960 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 16 MB 00:01 2023-10-07T07:43:07.343 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-07T07:43:07.760 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-07T07:43:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 454 kB/s | 24 kB 00:00 2023-10-07T07:43:11.141 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-07T07:43:11.466 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:13.756 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-07T07:43:13.756 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-07T07:43:14.039 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2023-10-07T07:43:14.040 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2023-10-07T07:43:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2023-10-07T07:43:14.040 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2023-10-07T07:43:14.040 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2023-10-07T07:43:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-07T07:43:14.041 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2023-10-07T07:43:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-07T07:43:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-07T07:43:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:43:14.041 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2023-10-07T07:43:14.041 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2023-10-07T07:43:14.041 INFO:teuthology.orchestra.run.smithi007.stdout:Install 3 Packages 2023-10-07T07:43:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:43:14.041 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 88 M 2023-10-07T07:43:14.042 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 96 M 2023-10-07T07:43:14.042 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2023-10-07T07:43:14.727 INFO:teuthology.orchestra.run.smithi007.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 16 MB/s | 10 MB 00:00 2023-10-07T07:43:16.236 INFO:teuthology.orchestra.run.smithi007.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 16 MB/s | 35 MB 00:02 2023-10-07T07:43:16.653 INFO:teuthology.orchestra.run.smithi007.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 17 MB/s | 43 MB 00:02 2023-10-07T07:43:16.653 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2023-10-07T07:43:16.653 INFO:teuthology.orchestra.run.smithi007.stdout:Total 34 MB/s | 88 MB 00:02 2023-10-07T07:43:17.358 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2023-10-07T07:43:17.829 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2023-10-07T07:43:17.829 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2023-10-07T07:43:19.126 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2023-10-07T07:43:19.151 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2023-10-07T07:43:22.100 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2023-10-07T07:43:22.684 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-07T07:43:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-07T07:43:26.825 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-07T07:43:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-07T07:43:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-07T07:43:34.974 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:34.974 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:43:34.975 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2023-10-07T07:43:34.975 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-07T07:43:34.975 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-07T07:43:34.975 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:43:34.975 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2023-10-07T07:43:35.104 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2023-10-07T07:43:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-07T07:43:35.315 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-07T07:43:35.315 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-07T07:43:35.315 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-07T07:43:35.315 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-07T07:43:35.316 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-07T07:43:35.316 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi172.front.sepia.ceph.com, path=None, version=distro) 2023-10-07T07:43:35.316 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2023-10-07T07:43:35.525 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-07T07:43:35.525 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-07T07:43:35.525 DEBUG:teuthology.orchestra.run.smithi172:> sudo rpm -qi grub2-tools 2023-10-07T07:43:35.559 INFO:teuthology.orchestra.run.smithi172.stdout:Name : grub2-tools 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Epoch : 1 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Version : 2.02 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Release : 129.el8 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Architecture: x86_64 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Group : System Environment/Base 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Size : 9265586 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:License : GPLv3+ 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-07T07:43:35.560 INFO:teuthology.orchestra.run.smithi172.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout:Relocations : (not relocatable) 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout:Packager : CentOS BuildSystem 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout:Vendor : CentOS 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout:Summary : Support tools for GRUB. 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout:Description : 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-07T07:43:35.561 INFO:teuthology.orchestra.run.smithi172.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-07T07:43:35.562 INFO:teuthology.orchestra.run.smithi172.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-07T07:43:35.562 INFO:teuthology.orchestra.run.smithi172.stdout:hardware devices. 2023-10-07T07:43:35.562 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:43:35.562 INFO:teuthology.orchestra.run.smithi172.stdout:This subpackage provides tools for support of all platforms. 2023-10-07T07:43:35.563 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-07T07:43:35.563 INFO:teuthology.task.kernel:Updating grub on smithi172 to boot 4.18.0-514.el8.x86_64 2023-10-07T07:43:35.563 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-07T07:43:36.986 INFO:teuthology.orchestra.run.smithi172.stderr:Generating grub configuration file ... 2023-10-07T07:43:38.941 INFO:teuthology.orchestra.run.smithi172.stderr:done 2023-10-07T07:43:38.943 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2023-10-07T07:43:38.966 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.2OrZgTs18X 2023-10-07T07:43:38.966 DEBUG:teuthology.orchestra.run.smithi172:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2OrZgTs18X 2023-10-07T07:43:39.041 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.2OrZgTs18X 2023-10-07T07:43:39.198 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.2OrZgTs18X is 6KB 2023-10-07T07:43:39.247 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.2OrZgTs18X 2023-10-07T07:43:39.265 DEBUG:teuthology.orchestra.run.smithi172:> sudo /bin/ls /boot/loader/entries || true 2023-10-07T07:43:39.346 INFO:teuthology.orchestra.run.smithi172.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-07T07:43:39.346 INFO:teuthology.orchestra.run.smithi172.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-07T07:43:39.346 INFO:teuthology.orchestra.run.smithi172.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-07T07:43:39.346 INFO:teuthology.orchestra.run.smithi172.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-07T07:43:39.347 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-07T07:43:39.509 DEBUG:teuthology.orchestra.run.smithi172:> sudo shutdown -r now 2023-10-07T07:43:54.502 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:54.528 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:54.973 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:54.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-07T07:43:54.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-07T07:43:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-07T07:43:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:43:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2023-10-07T07:43:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-07T07:43:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-07T07:43:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:43:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2023-10-07T07:43:55.862 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2023-10-07T07:43:56.087 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-07T07:43:56.087 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-07T07:43:56.087 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-07T07:43:56.087 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-07T07:43:56.087 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-07T07:43:56.087 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-07T07:43:56.087 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2023-10-07T07:43:56.088 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2023-10-07T07:43:56.305 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-07T07:43:56.305 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-07T07:43:56.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2023-10-07T07:43:56.340 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2023-10-07T07:43:56.340 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2023-10-07T07:43:56.340 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.02 2023-10-07T07:43:56.340 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 129.el8 2023-10-07T07:43:56.340 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2023-10-07T07:43:56.340 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-07T07:43:56.340 INFO:teuthology.orchestra.run.smithi007.stdout:Group : System Environment/Base 2023-10-07T07:43:56.340 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 9265586 2023-10-07T07:43:56.340 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:Relocations : (not relocatable) 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : CentOS BuildSystem 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:43:56.341 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-07T07:43:56.342 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-07T07:43:56.342 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-07T07:43:56.342 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2023-10-07T07:43:56.342 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:43:56.342 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2023-10-07T07:43:56.343 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-07T07:43:56.343 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 4.18.0-514.el8.x86_64 2023-10-07T07:43:56.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-07T07:43:57.825 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2023-10-07T07:43:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:done 2023-10-07T07:43:59.505 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2023-10-07T07:43:59.540 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.hu4Je6w4D8 2023-10-07T07:43:59.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hu4Je6w4D8 2023-10-07T07:43:59.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.hu4Je6w4D8 2023-10-07T07:43:59.762 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.hu4Je6w4D8 is 6KB 2023-10-07T07:43:59.785 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.hu4Je6w4D8 2023-10-07T07:43:59.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2023-10-07T07:43:59.869 INFO:teuthology.orchestra.run.smithi007.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-07T07:43:59.869 INFO:teuthology.orchestra.run.smithi007.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-07T07:43:59.870 INFO:teuthology.orchestra.run.smithi007.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-07T07:43:59.870 INFO:teuthology.orchestra.run.smithi007.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-07T07:43:59.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-07T07:44:00.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2023-10-07T07:44:09.518 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-07T07:44:09.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-10-07T07:44:09.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:44:28.026 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-10-07T07:44:30.014 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-07T07:44:30.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-07T07:44:30.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:44:37.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-10-07T07:44:37.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:44:40.090 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-10-07T07:44:48.506 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2023-10-07T07:44:52.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-10-07T07:44:52.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:44:57.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-07T07:44:57.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:45:10.525 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-10-07T07:45:25.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-10-07T07:45:25.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:45:25.816 DEBUG:teuthology.orchestra.run.smithi172:> true 2023-10-07T07:45:26.129 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-10-07T07:45:26.129 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-514.el8.x86_64"... 2023-10-07T07:45:26.129 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2023-10-07T07:45:26.148 INFO:teuthology.orchestra.run.smithi172.stdout:4.18.0-514.el8.x86_64 2023-10-07T07:45:26.148 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-07T07:45:26.148 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-07T07:45:26.148 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-07T07:45:27.149 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-07T07:45:27.150 DEBUG:teuthology.orchestra.run.smithi172:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-07T07:45:27.241 INFO:teuthology.orchestra.run.smithi172.stdout:ttyS1 2023-10-07T07:45:27.251 DEBUG:teuthology.parallel:result is None 2023-10-07T07:45:57.513 DEBUG:teuthology.orchestra.remote:timed out 2023-10-07T07:46:09.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-07T07:46:09.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:46:09.895 DEBUG:teuthology.orchestra.run.smithi007:> true 2023-10-07T07:46:10.197 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-07T07:46:10.197 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-514.el8.x86_64"... 2023-10-07T07:46:10.198 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2023-10-07T07:46:10.222 INFO:teuthology.orchestra.run.smithi007.stdout:4.18.0-514.el8.x86_64 2023-10-07T07:46:10.222 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-07T07:46:10.222 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-07T07:46:10.222 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-07T07:46:11.223 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-07T07:46:11.223 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-07T07:46:11.335 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2023-10-07T07:46:11.348 DEBUG:teuthology.parallel:result is None 2023-10-07T07:46:11.348 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-07T07:46:11.355 INFO:teuthology.task.internal:Creating test directory... 2023-10-07T07:46:11.355 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-07T07:46:11.358 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-07T07:46:11.383 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-07T07:46:11.420 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-07T07:46:11.430 INFO:teuthology.task.internal:Creating archive directory... 2023-10-07T07:46:11.430 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-07T07:46:11.433 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-07T07:46:11.465 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-07T07:46:11.473 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-07T07:46:11.473 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-07T07:46:11.502 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-07T07:46:11.542 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T07:46:11.544 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T07:46:11.553 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T07:46:11.556 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T07:46:11.558 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-07T07:46:11.567 INFO:teuthology.task.internal:Configuring sudo... 2023-10-07T07:46:11.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-07T07:46:11.569 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-07T07:46:11.625 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-07T07:46:11.634 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-07T07:46:11.634 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-07T07:46:11.644 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-07T07:46:11.684 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T07:46:11.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T07:46:11.817 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T07:46:11.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T07:46:11.917 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:46:11.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-07T07:46:11.992 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T07:46:12.040 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T07:46:12.098 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T07:46:12.185 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T07:46:12.228 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:46:12.229 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-07T07:46:12.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2023-10-07T07:46:12.300 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2023-10-07T07:46:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T07:46:12.413 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T07:46:12.821 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-07T07:46:12.828 INFO:teuthology.task.internal:Starting timer... 2023-10-07T07:46:12.829 INFO:teuthology.run_tasks:Running task pcp... 2023-10-07T07:46:12.838 INFO:teuthology.run_tasks:Running task selinux... 2023-10-07T07:46:12.846 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-07T07:46:12.846 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2023-10-07T07:46:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2023-10-07T07:46:12.958 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2023-10-07T07:46:13.045 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2023-10-07T07:46:13.047 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-07T07:46:13.047 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2023-10-07T07:46:13.107 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2023-10-07T07:46:13.107 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2023-10-07T07:46:13.168 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2023-10-07T07:46:13.168 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2023-10-07T07:46:13.168 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T07:46:13.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T07:46:13.205 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T07:46:13.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T07:46:13.233 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-07T07:46:13.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2023-10-07T07:46:13.280 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2023-10-07T07:46:13.308 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-07T07:46:13.317 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-07T07:46:13.317 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-07T07:46:13.335 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-07T07:46:13.336 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-07T07:54:24.938 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2023-10-07T07:54:24.939 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-07T07:54:24.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:54:25.011 DEBUG:teuthology.orchestra.run.smithi007:> true 2023-10-07T07:54:25.090 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-10-07T07:54:25.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-10-07T07:54:25.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T07:54:25.159 DEBUG:teuthology.orchestra.run.smithi172:> true 2023-10-07T07:54:25.229 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-10-07T07:54:25.229 INFO:teuthology.run_tasks:Running task clock... 2023-10-07T07:54:25.243 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-07T07:54:25.243 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-07T07:54:25.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T07:54:25.245 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-07T07:54:25.246 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T07:54:25.284 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-07T07:54:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-07T07:54:25.302 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-07T07:54:25.316 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-07T07:54:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2023-10-07T07:54:25.380 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2023-10-07T07:54:25.384 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2023-10-07T07:54:25.391 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2023-10-07T07:54:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-07T07:54:25.405 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-07T07:54:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-07T07:54:25.419 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-07T07:54:25.492 INFO:teuthology.orchestra.run.smithi172.stderr:bash: ntpq: command not found 2023-10-07T07:54:25.496 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T07:54:25.496 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2023-10-07T07:54:25.496 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T07:54:25.496 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T07:54:25.496 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T07:54:25.496 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T07:54:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2023-10-07T07:54:25.511 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T07:54:25.511 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2023-10-07T07:54:25.511 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T07:54:25.511 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T07:54:25.511 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T07:54:25.511 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T07:54:25.511 INFO:teuthology.run_tasks:Running task install... 2023-10-07T07:54:25.520 DEBUG:teuthology.task.install:project ceph 2023-10-07T07:54:25.520 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}} 2023-10-07T07:54:25.520 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'} 2023-10-07T07:54:25.520 INFO:teuthology.task.install:Using flavor: default 2023-10-07T07:54:25.526 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-07T07:54:25.526 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-07T07:54:25.526 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-07T07:54:25.526 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T07:54:25.528 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-07T07:54:25.528 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T07:54:25.688 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-07T07:54:25.688 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-07T07:54:25.806 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-07T07:54:25.806 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-07T07:54:25.925 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-07T07:54:25.925 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:54:25.925 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-07T07:54:25.955 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-07T07:54:25.956 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-07T07:54:25.982 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-07T07:54:25.982 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:54:25.983 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-07T07:54:26.019 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-07T07:54:26.019 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-07T07:54:26.024 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-07T07:54:26.096 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-07T07:54:26.103 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-07T07:54:26.170 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2023-10-07T07:54:26.172 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2023-10-07T07:54:26.185 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-07T07:54:26.217 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2023-10-07T07:54:26.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2023-10-07T07:54:26.437 INFO:teuthology.orchestra.run.smithi172.stdout:76 files removed 2023-10-07T07:54:26.465 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-07T07:54:26.540 INFO:teuthology.orchestra.run.smithi007.stdout:76 files removed 2023-10-07T07:54:26.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-07T07:54:27.672 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 98 kB/s | 78 kB 00:00 2023-10-07T07:54:27.890 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 92 kB/s | 78 kB 00:00 2023-10-07T07:54:28.215 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 18 kB/s | 9.2 kB 00:00 2023-10-07T07:54:28.491 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 17 kB/s | 9.2 kB 00:00 2023-10-07T07:54:28.957 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 2.0 kB/s | 1.5 kB 00:00 2023-10-07T07:54:29.158 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 2.3 kB/s | 1.5 kB 00:00 2023-10-07T07:54:31.024 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 8 - AppStream 16 MB/s | 33 MB 00:02 2023-10-07T07:54:32.000 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 8 - AppStream 12 MB/s | 33 MB 00:02 2023-10-07T07:54:39.208 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 8 - BaseOS 23 MB/s | 48 MB 00:02 2023-10-07T07:54:40.825 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 8 - BaseOS 27 MB/s | 48 MB 00:01 2023-10-07T07:54:46.491 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 8 - Extras 47 kB/s | 18 kB 00:00 2023-10-07T07:54:47.041 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 8 - Extras common packages 19 kB/s | 6.9 kB 00:00 2023-10-07T07:54:48.118 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 8 - PowerTools 7.0 MB/s | 6.5 MB 00:00 2023-10-07T07:54:49.109 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 8 - Extras 63 kB/s | 18 kB 00:00 2023-10-07T07:54:49.519 INFO:teuthology.orchestra.run.smithi172.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-10-07T07:54:49.701 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 8 - Extras common packages 18 kB/s | 6.9 kB 00:00 2023-10-07T07:54:50.893 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 8 - PowerTools 6.4 MB/s | 6.5 MB 00:01 2023-10-07T07:54:52.161 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 16 MB 00:02 2023-10-07T07:54:52.513 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-10-07T07:54:53.508 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2023-10-07T07:54:56.003 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 402 kB/s | 24 kB 00:00 2023-10-07T07:54:57.997 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 300 kB/s | 24 kB 00:00 2023-10-07T07:55:00.176 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T07:55:00.177 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T07:55:00.191 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-07T07:55:00.191 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-07T07:55:00.192 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-07T07:55:00.193 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-07T07:55:00.195 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-07T07:55:00.197 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-07T07:55:00.200 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-07T07:55:00.201 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-07T07:55:00.416 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2023-10-07T07:55:00.429 INFO:teuthology.orchestra.run.smithi172.stdout:==================================================================================================== 2023-10-07T07:55:00.429 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2023-10-07T07:55:00.429 INFO:teuthology.orchestra.run.smithi172.stdout:==================================================================================================== 2023-10-07T07:55:00.429 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2023-10-07T07:55:00.429 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-07T07:55:00.429 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-07T07:55:00.430 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-07T07:55:00.430 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-07T07:55:00.430 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-07T07:55:00.430 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-07T07:55:00.430 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-07T07:55:00.430 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-07T07:55:00.430 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-07T07:55:00.430 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-07T07:55:00.430 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-07T07:55:00.431 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-07T07:55:00.432 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-07T07:55:00.432 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-10-07T07:55:00.432 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-10-07T07:55:00.432 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-07T07:55:00.432 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-07T07:55:00.432 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-07T07:55:00.432 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-07T07:55:00.432 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-07T07:55:00.432 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-07T07:55:00.433 INFO:teuthology.orchestra.run.smithi172.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-10-07T07:55:00.434 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-07T07:55:00.435 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-07T07:55:00.436 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-07T07:55:00.437 INFO:teuthology.orchestra.run.smithi172.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-07T07:55:00.438 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-07T07:55:00.439 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-07T07:55:00.440 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-07T07:55:00.441 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-07T07:55:00.442 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout:==================================================================================================== 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout:Install 123 Packages 2023-10-07T07:55:00.443 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 16 Packages 2023-10-07T07:55:00.444 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:55:00.444 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 210 M 2023-10-07T07:55:00.444 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2023-10-07T07:55:01.454 INFO:teuthology.orchestra.run.smithi172.stdout:(1/139): ceph-17.2.6-1320.g08cac018.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2023-10-07T07:55:01.945 INFO:teuthology.orchestra.run.smithi172.stdout:(2/139): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 1.7 MB/s | 876 kB 00:00 2023-10-07T07:55:02.079 INFO:teuthology.orchestra.run.smithi172.stdout:(3/139): ceph-immutable-object-cache-17.2.6-132 1.2 MB/s | 162 kB 00:00 2023-10-07T07:55:02.683 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T07:55:02.684 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T07:55:02.700 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-07T07:55:02.700 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-07T07:55:02.701 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-07T07:55:02.702 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-07T07:55:02.703 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-07T07:55:02.705 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-07T07:55:02.709 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-07T07:55:02.710 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-07T07:55:02.954 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2023-10-07T07:55:02.969 INFO:teuthology.orchestra.run.smithi007.stdout:==================================================================================================== 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout:==================================================================================================== 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-07T07:55:02.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-10-07T07:55:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T07:55:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T07:55:02.973 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-07T07:55:02.974 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-07T07:55:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-07T07:55:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-07T07:55:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-07T07:55:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-07T07:55:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-07T07:55:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-07T07:55:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:55:02.982 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2023-10-07T07:55:02.982 INFO:teuthology.orchestra.run.smithi007.stdout:==================================================================================================== 2023-10-07T07:55:02.982 INFO:teuthology.orchestra.run.smithi007.stdout:Install 123 Packages 2023-10-07T07:55:02.982 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 16 Packages 2023-10-07T07:55:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:55:02.982 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 210 M 2023-10-07T07:55:02.982 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2023-10-07T07:55:03.281 INFO:teuthology.orchestra.run.smithi172.stdout:(4/139): ceph-base-17.2.6-1320.g08cac018.el8.x8 2.4 MB/s | 5.2 MB 00:02 2023-10-07T07:55:03.379 INFO:teuthology.orchestra.run.smithi007.stdout:(1/139): ceph-17.2.6-1320.g08cac018.el8.x86_64. 27 kB/s | 6.4 kB 00:00 2023-10-07T07:55:03.755 INFO:teuthology.orchestra.run.smithi172.stdout:(5/139): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 3.3 MB/s | 1.6 MB 00:00 2023-10-07T07:55:03.805 INFO:teuthology.orchestra.run.smithi172.stdout:(6/139): ceph-mds-17.2.6-1320.g08cac018.el8.x86 1.3 MB/s | 2.3 MB 00:01 2023-10-07T07:55:04.323 INFO:teuthology.orchestra.run.smithi007.stdout:(2/139): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 928 kB/s | 876 kB 00:00 2023-10-07T07:55:04.523 INFO:teuthology.orchestra.run.smithi007.stdout:(3/139): ceph-immutable-object-cache-17.2.6-132 809 kB/s | 162 kB 00:00 2023-10-07T07:55:05.163 INFO:teuthology.orchestra.run.smithi172.stdout:(7/139): ceph-mon-17.2.6-1320.g08cac018.el8.x86 2.8 MB/s | 3.9 MB 00:01 2023-10-07T07:55:05.715 INFO:teuthology.orchestra.run.smithi007.stdout:(4/139): ceph-mds-17.2.6-1320.g08cac018.el8.x86 1.9 MB/s | 2.3 MB 00:01 2023-10-07T07:55:05.924 INFO:teuthology.orchestra.run.smithi007.stdout:(5/139): ceph-base-17.2.6-1320.g08cac018.el8.x8 1.9 MB/s | 5.2 MB 00:02 2023-10-07T07:55:06.224 INFO:teuthology.orchestra.run.smithi007.stdout:(6/139): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 3.1 MB/s | 1.6 MB 00:00 2023-10-07T07:55:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:(7/139): ceph-mon-17.2.6-1320.g08cac018.el8.x86 4.6 MB/s | 3.9 MB 00:00 2023-10-07T07:55:09.391 INFO:teuthology.orchestra.run.smithi007.stdout:(8/139): ceph-radosgw-17.2.6-1320.g08cac018.el8 6.4 MB/s | 17 MB 00:02 2023-10-07T07:55:09.530 INFO:teuthology.orchestra.run.smithi172.stdout:(8/139): ceph-common-17.2.6-1320.g08cac018.el8. 2.8 MB/s | 23 MB 00:08 2023-10-07T07:55:09.575 INFO:teuthology.orchestra.run.smithi007.stdout:(9/139): ceph-osd-17.2.6-1320.g08cac018.el8.x86 5.0 MB/s | 17 MB 00:03 2023-10-07T07:55:09.580 INFO:teuthology.orchestra.run.smithi172.stdout:(9/139): ceph-selinux-17.2.6-1320.g08cac018.el8 491 kB/s | 24 kB 00:00 2023-10-07T07:55:09.600 INFO:teuthology.orchestra.run.smithi007.stdout:(10/139): ceph-selinux-17.2.6-1320.g08cac018.el 117 kB/s | 24 kB 00:00 2023-10-07T07:55:09.684 INFO:teuthology.orchestra.run.smithi007.stdout:(11/139): cephfs-mirror-17.2.6-1320.g08cac018.e 3.0 MB/s | 255 kB 00:00 2023-10-07T07:55:09.742 INFO:teuthology.orchestra.run.smithi007.stdout:(12/139): libcephfs-devel-17.2.6-1320.g08cac018 426 kB/s | 25 kB 00:00 2023-10-07T07:55:09.893 INFO:teuthology.orchestra.run.smithi007.stdout:(13/139): libcephfs2-17.2.6-1320.g08cac018.el8. 5.2 MB/s | 797 kB 00:00 2023-10-07T07:55:09.960 INFO:teuthology.orchestra.run.smithi007.stdout:(14/139): libcephsqlite-17.2.6-1320.g08cac018.e 2.6 MB/s | 174 kB 00:00 2023-10-07T07:55:10.010 INFO:teuthology.orchestra.run.smithi007.stdout:(15/139): librados-devel-17.2.6-1320.g08cac018. 2.6 MB/s | 130 kB 00:00 2023-10-07T07:55:10.111 INFO:teuthology.orchestra.run.smithi007.stdout:(16/139): libradosstriper1-17.2.6-1320.g08cac01 4.9 MB/s | 507 kB 00:00 2023-10-07T07:55:10.969 INFO:teuthology.orchestra.run.smithi007.stdout:(17/139): ceph-common-17.2.6-1320.g08cac018.el8 3.0 MB/s | 23 MB 00:07 2023-10-07T07:55:11.038 INFO:teuthology.orchestra.run.smithi172.stdout:(10/139): ceph-radosgw-17.2.6-1320.g08cac018.el 2.9 MB/s | 17 MB 00:05 2023-10-07T07:55:11.079 INFO:teuthology.orchestra.run.smithi007.stdout:(18/139): librgw2-17.2.6-1320.g08cac018.el8.x86 5.7 MB/s | 5.6 MB 00:00 2023-10-07T07:55:11.111 INFO:teuthology.orchestra.run.smithi007.stdout:(19/139): python3-ceph-argparse-17.2.6-1320.g08 315 kB/s | 45 kB 00:00 2023-10-07T07:55:11.113 INFO:teuthology.orchestra.run.smithi172.stdout:(11/139): cephfs-mirror-17.2.6-1320.g08cac018.e 3.3 MB/s | 255 kB 00:00 2023-10-07T07:55:11.136 INFO:teuthology.orchestra.run.smithi007.stdout:(20/139): python3-ceph-common-17.2.6-1320.g08ca 2.0 MB/s | 115 kB 00:00 2023-10-07T07:55:11.164 INFO:teuthology.orchestra.run.smithi172.stdout:(12/139): libcephfs-devel-17.2.6-1320.g08cac018 498 kB/s | 25 kB 00:00 2023-10-07T07:55:11.220 INFO:teuthology.orchestra.run.smithi007.stdout:(21/139): python3-rados-17.2.6-1320.g08cac018.e 4.4 MB/s | 371 kB 00:00 2023-10-07T07:55:11.245 INFO:teuthology.orchestra.run.smithi007.stdout:(22/139): python3-cephfs-17.2.6-1320.g08cac018. 1.4 MB/s | 190 kB 00:00 2023-10-07T07:55:11.312 INFO:teuthology.orchestra.run.smithi007.stdout:(23/139): python3-rbd-17.2.6-1320.g08cac018.el8 3.8 MB/s | 358 kB 00:00 2023-10-07T07:55:11.356 INFO:teuthology.orchestra.run.smithi172.stdout:(13/139): libcephfs2-17.2.6-1320.g08cac018.el8. 4.1 MB/s | 797 kB 00:00 2023-10-07T07:55:11.374 INFO:teuthology.orchestra.run.smithi007.stdout:(24/139): python3-rgw-17.2.6-1320.g08cac018.el8 833 kB/s | 107 kB 00:00 2023-10-07T07:55:11.397 INFO:teuthology.orchestra.run.smithi007.stdout:(25/139): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 1.1 MB/s | 90 kB 00:00 2023-10-07T07:55:11.455 INFO:teuthology.orchestra.run.smithi007.stdout:(26/139): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 2.9 MB/s | 173 kB 00:00 2023-10-07T07:55:11.464 INFO:teuthology.orchestra.run.smithi172.stdout:(14/139): libcephsqlite-17.2.6-1320.g08cac018.e 1.6 MB/s | 174 kB 00:00 2023-10-07T07:55:11.489 INFO:teuthology.orchestra.run.smithi007.stdout:(27/139): ceph-grafana-dashboards-17.2.6-1320.g 690 kB/s | 23 kB 00:00 2023-10-07T07:55:11.523 INFO:teuthology.orchestra.run.smithi172.stdout:(15/139): librados-devel-17.2.6-1320.g08cac018. 2.2 MB/s | 130 kB 00:00 2023-10-07T07:55:11.539 INFO:teuthology.orchestra.run.smithi007.stdout:(28/139): ceph-mgr-cephadm-17.2.6-1320.g08cac01 2.4 MB/s | 121 kB 00:00 2023-10-07T07:55:11.715 INFO:teuthology.orchestra.run.smithi172.stdout:(16/139): ceph-osd-17.2.6-1320.g08cac018.el8.x8 2.1 MB/s | 17 MB 00:07 2023-10-07T07:55:11.773 INFO:teuthology.orchestra.run.smithi172.stdout:(17/139): libradosstriper1-17.2.6-1320.g08cac01 2.0 MB/s | 507 kB 00:00 2023-10-07T07:55:11.806 INFO:teuthology.orchestra.run.smithi007.stdout:(29/139): ceph-mgr-dashboard-17.2.6-1320.g08cac 6.4 MB/s | 1.7 MB 00:00 2023-10-07T07:55:11.815 INFO:teuthology.orchestra.run.smithi172.stdout:(18/139): python3-ceph-argparse-17.2.6-1320.g08 1.0 MB/s | 45 kB 00:00 2023-10-07T07:55:11.874 INFO:teuthology.orchestra.run.smithi172.stdout:(19/139): python3-ceph-common-17.2.6-1320.g08ca 1.9 MB/s | 115 kB 00:00 2023-10-07T07:55:12.022 INFO:teuthology.orchestra.run.smithi172.stdout:(20/139): python3-cephfs-17.2.6-1320.g08cac018. 1.3 MB/s | 190 kB 00:00 2023-10-07T07:55:12.082 INFO:teuthology.orchestra.run.smithi007.stdout:(30/139): rbd-mirror-17.2.6-1320.g08cac018.el8. 4.9 MB/s | 3.5 MB 00:00 2023-10-07T07:55:12.142 INFO:teuthology.orchestra.run.smithi172.stdout:(21/139): python3-rados-17.2.6-1320.g08cac018.e 3.0 MB/s | 371 kB 00:00 2023-10-07T07:55:12.165 INFO:teuthology.orchestra.run.smithi007.stdout:(31/139): ceph-mgr-modules-core-17.2.6-1320.g08 2.8 MB/s | 238 kB 00:00 2023-10-07T07:55:12.283 INFO:teuthology.orchestra.run.smithi172.stdout:(22/139): python3-rbd-17.2.6-1320.g08cac018.el8 2.5 MB/s | 358 kB 00:00 2023-10-07T07:55:12.290 INFO:teuthology.orchestra.run.smithi007.stdout:(32/139): ceph-mgr-rook-17.2.6-1320.g08cac018.e 379 kB/s | 47 kB 00:00 2023-10-07T07:55:12.342 INFO:teuthology.orchestra.run.smithi172.stdout:(23/139): python3-rgw-17.2.6-1320.g08cac018.el8 1.8 MB/s | 107 kB 00:00 2023-10-07T07:55:12.366 INFO:teuthology.orchestra.run.smithi007.stdout:(33/139): ceph-prometheus-alerts-17.2.6-1320.g0 193 kB/s | 14 kB 00:00 2023-10-07T07:55:12.401 INFO:teuthology.orchestra.run.smithi172.stdout:(24/139): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 1.5 MB/s | 90 kB 00:00 2023-10-07T07:55:12.458 INFO:teuthology.orchestra.run.smithi007.stdout:(34/139): ceph-volume-17.2.6-1320.g08cac018.el8 2.8 MB/s | 267 kB 00:00 2023-10-07T07:55:12.556 INFO:teuthology.orchestra.run.smithi007.stdout:(35/139): cephadm-17.2.6-1320.g08cac018.el8.noa 826 kB/s | 80 kB 00:00 2023-10-07T07:55:12.642 INFO:teuthology.orchestra.run.smithi007.stdout:(36/139): cephfs-top-17.2.6-1320.g08cac018.el8. 210 kB/s | 18 kB 00:00 2023-10-07T07:55:12.734 INFO:teuthology.orchestra.run.smithi007.stdout:(37/139): ceph-mgr-diskprediction-local-17.2.6- 8.0 MB/s | 7.4 MB 00:00 2023-10-07T07:55:12.943 INFO:teuthology.orchestra.run.smithi007.stdout:(38/139): automake-1.16.1-8.el8.noarch.rpm 2.4 MB/s | 740 kB 00:00 2023-10-07T07:55:13.010 INFO:teuthology.orchestra.run.smithi007.stdout:(39/139): bison-3.0.4-10.el8.x86_64.rpm 2.4 MB/s | 688 kB 00:00 2023-10-07T07:55:13.035 INFO:teuthology.orchestra.run.smithi007.stdout:(40/139): flex-2.6.1-9.el8.x86_64.rpm 3.4 MB/s | 320 kB 00:00 2023-10-07T07:55:13.060 INFO:teuthology.orchestra.run.smithi007.stdout:(41/139): jq-1.6-7.el8.x86_64.rpm 3.9 MB/s | 202 kB 00:00 2023-10-07T07:55:13.094 INFO:teuthology.orchestra.run.smithi007.stdout:(42/139): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-10-07T07:55:13.126 INFO:teuthology.orchestra.run.smithi172.stdout:(25/139): librgw2-17.2.6-1320.g08cac018.el8.x86 3.9 MB/s | 5.6 MB 00:01 2023-10-07T07:55:13.169 INFO:teuthology.orchestra.run.smithi007.stdout:(43/139): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2023-10-07T07:55:13.195 INFO:teuthology.orchestra.run.smithi007.stdout:(44/139): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.5 MB/s | 259 kB 00:00 2023-10-07T07:55:13.212 INFO:teuthology.orchestra.run.smithi007.stdout:(45/139): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-10-07T07:55:13.218 INFO:teuthology.orchestra.run.smithi172.stdout:(26/139): rbd-mirror-17.2.6-1320.g08cac018.el8. 4.2 MB/s | 3.5 MB 00:00 2023-10-07T07:55:13.243 INFO:teuthology.orchestra.run.smithi172.stdout:(27/139): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 1.4 MB/s | 173 kB 00:00 2023-10-07T07:55:13.269 INFO:teuthology.orchestra.run.smithi172.stdout:(28/139): ceph-grafana-dashboards-17.2.6-1320.g 461 kB/s | 23 kB 00:00 2023-10-07T07:55:13.294 INFO:teuthology.orchestra.run.smithi172.stdout:(29/139): ceph-mgr-cephadm-17.2.6-1320.g08cac01 2.3 MB/s | 121 kB 00:00 2023-10-07T07:55:13.437 INFO:teuthology.orchestra.run.smithi007.stdout:(46/139): openblas-threads-0.3.15-4.el8.x86_64. 22 MB/s | 4.9 MB 00:00 2023-10-07T07:55:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:(47/139): openblas-0.3.15-4.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-10-07T07:55:13.552 INFO:teuthology.orchestra.run.smithi172.stdout:(30/139): ceph-mgr-dashboard-17.2.6-1320.g08cac 6.0 MB/s | 1.7 MB 00:00 2023-10-07T07:55:13.580 INFO:teuthology.orchestra.run.smithi007.stdout:(48/139): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-10-07T07:55:13.619 INFO:teuthology.orchestra.run.smithi172.stdout:(31/139): ceph-mgr-modules-core-17.2.6-1320.g08 3.5 MB/s | 238 kB 00:00 2023-10-07T07:55:13.670 INFO:teuthology.orchestra.run.smithi172.stdout:(32/139): ceph-mgr-rook-17.2.6-1320.g08cac018.e 944 kB/s | 47 kB 00:00 2023-10-07T07:55:13.695 INFO:teuthology.orchestra.run.smithi007.stdout:(49/139): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-10-07T07:55:13.722 INFO:teuthology.orchestra.run.smithi007.stdout:(50/139): python3-jsonpatch-1.21-2.el8.noarch.r 187 kB/s | 27 kB 00:00 2023-10-07T07:55:13.739 INFO:teuthology.orchestra.run.smithi007.stdout:(51/139): python3-jsonpointer-1.10-11.el8.noarc 464 kB/s | 20 kB 00:00 2023-10-07T07:55:13.749 INFO:teuthology.orchestra.run.smithi172.stdout:(33/139): ceph-prometheus-alerts-17.2.6-1320.g0 184 kB/s | 14 kB 00:00 2023-10-07T07:55:13.772 INFO:teuthology.orchestra.run.smithi007.stdout:(52/139): python3-mako-1.0.6-14.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-10-07T07:55:13.789 INFO:teuthology.orchestra.run.smithi007.stdout:(53/139): python3-markupsafe-0.23-19.el8.x86_64 769 kB/s | 39 kB 00:00 2023-10-07T07:55:13.820 INFO:teuthology.orchestra.run.smithi172.stdout:(34/139): ceph-volume-17.2.6-1320.g08cac018.el8 3.7 MB/s | 267 kB 00:00 2023-10-07T07:55:13.848 INFO:teuthology.orchestra.run.smithi007.stdout:(54/139): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.8 MB/s | 225 kB 00:00 2023-10-07T07:55:13.862 INFO:teuthology.orchestra.run.smithi172.stdout:(35/139): cephadm-17.2.6-1320.g08cac018.el8.noa 1.9 MB/s | 80 kB 00:00 2023-10-07T07:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:(55/139): python3-prettytable-0.7.2-14.el8.noar 881 kB/s | 44 kB 00:00 2023-10-07T07:55:13.913 INFO:teuthology.orchestra.run.smithi172.stdout:(36/139): cephfs-top-17.2.6-1320.g08cac018.el8. 360 kB/s | 18 kB 00:00 2023-10-07T07:55:13.993 INFO:teuthology.orchestra.run.smithi007.stdout:(56/139): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-10-07T07:55:14.024 INFO:teuthology.orchestra.run.smithi007.stdout:(57/139): python3-pyOpenSSL-19.0.0-1.el8.noarch 826 kB/s | 103 kB 00:00 2023-10-07T07:55:14.050 INFO:teuthology.orchestra.run.smithi007.stdout:(58/139): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.2 MB/s | 126 kB 00:00 2023-10-07T07:55:14.075 INFO:teuthology.orchestra.run.smithi007.stdout:(59/139): python3-pyasn1-modules-0.3.7-6.el8.no 2.1 MB/s | 110 kB 00:00 2023-10-07T07:55:14.100 INFO:teuthology.orchestra.run.smithi007.stdout:(60/139): python3-pytz-2017.2-11.el8.noarch.rpm 1.1 MB/s | 57 kB 00:00 2023-10-07T07:55:14.167 INFO:teuthology.orchestra.run.smithi007.stdout:(61/139): python3-werkzeug-0.12.2-4.el8.noarch. 6.7 MB/s | 457 kB 00:00 2023-10-07T07:55:14.171 INFO:teuthology.orchestra.run.smithi172.stdout:(37/139): automake-1.16.1-8.el8.noarch.rpm 2.8 MB/s | 740 kB 00:00 2023-10-07T07:55:14.234 INFO:teuthology.orchestra.run.smithi007.stdout:(62/139): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-10-07T07:55:14.280 INFO:teuthology.orchestra.run.smithi172.stdout:(38/139): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2023-10-07T07:55:14.301 INFO:teuthology.orchestra.run.smithi007.stdout:(63/139): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2023-10-07T07:55:14.355 INFO:teuthology.orchestra.run.smithi172.stdout:(39/139): flex-2.6.1-9.el8.x86_64.rpm 4.2 MB/s | 320 kB 00:00 2023-10-07T07:55:14.405 INFO:teuthology.orchestra.run.smithi172.stdout:(40/139): jq-1.6-7.el8.x86_64.rpm 3.9 MB/s | 202 kB 00:00 2023-10-07T07:55:14.464 INFO:teuthology.orchestra.run.smithi172.stdout:(41/139): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-10-07T07:55:14.510 INFO:teuthology.orchestra.run.smithi007.stdout:(64/139): python3-scipy-1.0.0-21.module_el8.5.0 31 MB/s | 14 MB 00:00 2023-10-07T07:55:14.566 INFO:teuthology.orchestra.run.smithi007.stdout:(65/139): elfutils-libelf-devel-0.189-3.el8.x86 233 kB/s | 62 kB 00:00 2023-10-07T07:55:14.573 INFO:teuthology.orchestra.run.smithi172.stdout:(42/139): ceph-mgr-diskprediction-local-17.2.6- 5.8 MB/s | 7.4 MB 00:01 2023-10-07T07:55:14.606 INFO:teuthology.orchestra.run.smithi172.stdout:(43/139): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2023-10-07T07:55:14.644 INFO:teuthology.orchestra.run.smithi007.stdout:(66/139): keyutils-libs-devel-1.5.10-9.el8.x86_ 623 kB/s | 48 kB 00:00 2023-10-07T07:55:14.665 INFO:teuthology.orchestra.run.smithi172.stdout:(44/139): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-10-07T07:55:14.711 INFO:teuthology.orchestra.run.smithi007.stdout:(67/139): fuse-2.9.7-17.el8.x86_64.rpm 415 kB/s | 83 kB 00:00 2023-10-07T07:55:14.765 INFO:teuthology.orchestra.run.smithi172.stdout:(45/139): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2023-10-07T07:55:14.769 INFO:teuthology.orchestra.run.smithi007.stdout:(68/139): libaio-devel-0.3.112-1.el8.x86_64.rpm 321 kB/s | 19 kB 00:00 2023-10-07T07:55:14.845 INFO:teuthology.orchestra.run.smithi007.stdout:(69/139): krb5-devel-1.18.2-22.el8.x86_64.rpm 2.7 MB/s | 561 kB 00:00 2023-10-07T07:55:14.878 INFO:teuthology.orchestra.run.smithi007.stdout:(70/139): libcom_err-devel-1.45.6-5.el8.x86_64. 356 kB/s | 39 kB 00:00 2023-10-07T07:55:14.937 INFO:teuthology.orchestra.run.smithi007.stdout:(71/139): libgfortran-8.5.0-20.el8.x86_64.rpm 6.9 MB/s | 645 kB 00:00 2023-10-07T07:55:14.974 INFO:teuthology.orchestra.run.smithi172.stdout:(46/139): openblas-0.3.15-4.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-10-07T07:55:14.995 INFO:teuthology.orchestra.run.smithi007.stdout:(72/139): librabbitmq-0.9.0-4.el8.x86_64.rpm 809 kB/s | 47 kB 00:00 2023-10-07T07:55:15.021 INFO:teuthology.orchestra.run.smithi007.stdout:(73/139): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2023-10-07T07:55:15.080 INFO:teuthology.orchestra.run.smithi007.stdout:(74/139): libselinux-devel-2.9-8.el8.x86_64.rpm 2.3 MB/s | 200 kB 00:00 2023-10-07T07:55:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:(75/139): libsepol-devel-2.9-3.el8.x86_64.rpm 743 kB/s | 87 kB 00:00 2023-10-07T07:55:15.164 INFO:teuthology.orchestra.run.smithi007.stdout:(76/139): libuuid-devel-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 100 kB 00:00 2023-10-07T07:55:15.230 INFO:teuthology.orchestra.run.smithi007.stdout:(77/139): libverto-devel-0.3.2-2.el8.x86_64.rpm 196 kB/s | 18 kB 00:00 2023-10-07T07:55:15.256 INFO:teuthology.orchestra.run.smithi007.stdout:(78/139): libzstd-devel-1.4.4-1.el8.x86_64.rpm 478 kB/s | 44 kB 00:00 2023-10-07T07:55:15.366 INFO:teuthology.orchestra.run.smithi172.stdout:(47/139): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-10-07T07:55:15.450 INFO:teuthology.orchestra.run.smithi172.stdout:(48/139): openblas-threads-0.3.15-4.el8.x86_64. 7.1 MB/s | 4.9 MB 00:00 2023-10-07T07:55:15.508 INFO:teuthology.orchestra.run.smithi172.stdout:(49/139): python3-jinja2-2.10.1-3.el8.noarch.rp 3.7 MB/s | 538 kB 00:00 2023-10-07T07:55:15.559 INFO:teuthology.orchestra.run.smithi172.stdout:(50/139): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-10-07T07:55:15.609 INFO:teuthology.orchestra.run.smithi172.stdout:(51/139): python3-mako-1.0.6-14.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-10-07T07:55:15.659 INFO:teuthology.orchestra.run.smithi172.stdout:(52/139): python3-markupsafe-0.23-19.el8.x86_64 773 kB/s | 39 kB 00:00 2023-10-07T07:55:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:(79/139): ceph-test-17.2.6-1320.g08cac018.el8.x 8.3 MB/s | 52 MB 00:06 2023-10-07T07:55:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:(80/139): pcre2-devel-10.32-3.el8.x86_64.rpm 1.1 MB/s | 605 kB 00:00 2023-10-07T07:55:15.835 INFO:teuthology.orchestra.run.smithi172.stdout:(53/139): python3-numpy-1.14.3-10.el8.x86_64.rp 21 MB/s | 3.7 MB 00:00 2023-10-07T07:55:15.860 INFO:teuthology.orchestra.run.smithi172.stdout:(54/139): python3-jsonpatch-1.21-2.el8.noarch.r 65 kB/s | 27 kB 00:00 2023-10-07T07:55:15.865 INFO:teuthology.orchestra.run.smithi007.stdout:(81/139): pcre2-utf32-10.32-3.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-10-07T07:55:15.893 INFO:teuthology.orchestra.run.smithi172.stdout:(55/139): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.8 MB/s | 225 kB 00:00 2023-10-07T07:55:15.919 INFO:teuthology.orchestra.run.smithi172.stdout:(56/139): python3-prettytable-0.7.2-14.el8.noar 750 kB/s | 44 kB 00:00 2023-10-07T07:55:15.932 INFO:teuthology.orchestra.run.smithi007.stdout:(82/139): python3-cffi-1.11.5-6.el8.x86_64.rpm 3.6 MB/s | 249 kB 00:00 2023-10-07T07:55:15.944 INFO:teuthology.orchestra.run.smithi172.stdout:(57/139): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-10-07T07:55:15.969 INFO:teuthology.orchestra.run.smithi172.stdout:(58/139): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-10-07T07:55:16.003 INFO:teuthology.orchestra.run.smithi172.stdout:(59/139): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-10-07T07:55:16.020 INFO:teuthology.orchestra.run.smithi172.stdout:(60/139): python3-pytz-2017.2-11.el8.noarch.rpm 1.1 MB/s | 57 kB 00:00 2023-10-07T07:55:16.032 INFO:teuthology.orchestra.run.smithi007.stdout:(83/139): python3-cryptography-3.2.1-6.el8.x86_ 6.3 MB/s | 641 kB 00:00 2023-10-07T07:55:16.058 INFO:teuthology.orchestra.run.smithi007.stdout:(84/139): pcre2-utf16-10.32-3.el8.x86_64.rpm 806 kB/s | 229 kB 00:00 2023-10-07T07:55:16.091 INFO:teuthology.orchestra.run.smithi007.stdout:(85/139): python3-jwt-1.6.1-2.el8.noarch.rpm 728 kB/s | 43 kB 00:00 2023-10-07T07:55:16.128 INFO:teuthology.orchestra.run.smithi172.stdout:(61/139): python3-werkzeug-0.12.2-4.el8.noarch. 4.1 MB/s | 457 kB 00:00 2023-10-07T07:55:16.158 INFO:teuthology.orchestra.run.smithi007.stdout:(86/139): openssl-devel-1.1.1k-9.el8.x86_64.rpm 3.4 MB/s | 3.2 MB 00:00 2023-10-07T07:55:16.184 INFO:teuthology.orchestra.run.smithi007.stdout:(87/139): python3-pycparser-2.14-14.el8.noarch. 1.2 MB/s | 109 kB 00:00 2023-10-07T07:55:16.196 INFO:teuthology.orchestra.run.smithi172.stdout:(62/139): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-10-07T07:55:16.209 INFO:teuthology.orchestra.run.smithi007.stdout:(88/139): python3-oauthlib-2.1.0-1.el8.noarch.r 1.0 MB/s | 155 kB 00:00 2023-10-07T07:55:16.226 INFO:teuthology.orchestra.run.smithi007.stdout:(89/139): python3-requests-oauthlib-1.0.0-1.el8 635 kB/s | 43 kB 00:00 2023-10-07T07:55:16.243 INFO:teuthology.orchestra.run.smithi007.stdout:(90/139): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 1.2 MB/s | 74 kB 00:00 2023-10-07T07:55:16.254 INFO:teuthology.orchestra.run.smithi172.stdout:(63/139): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2023-10-07T07:55:16.310 INFO:teuthology.orchestra.run.smithi007.stdout:(91/139): zlib-devel-1.2.11-21.el8.x86_64.rpm 577 kB/s | 58 kB 00:00 2023-10-07T07:55:16.518 INFO:teuthology.orchestra.run.smithi007.stdout:(92/139): liboath-2.6.2-3.el8.x86_64.rpm 214 kB/s | 59 kB 00:00 2023-10-07T07:55:16.594 INFO:teuthology.orchestra.run.smithi007.stdout:(93/139): libunwind-1.3.1-3.el8.x86_64.rpm 265 kB/s | 75 kB 00:00 2023-10-07T07:55:16.596 INFO:teuthology.orchestra.run.smithi172.stdout:(64/139): python3-scipy-1.0.0-21.module_el8.5.0 23 MB/s | 14 MB 00:00 2023-10-07T07:55:16.652 INFO:teuthology.orchestra.run.smithi007.stdout:(94/139): gperftools-libs-2.7-9.el8.x86_64.rpm 719 kB/s | 306 kB 00:00 2023-10-07T07:55:16.663 INFO:teuthology.orchestra.run.smithi172.stdout:(65/139): elfutils-libelf-devel-0.189-3.el8.x86 151 kB/s | 62 kB 00:00 2023-10-07T07:55:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:(95/139): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 523 kB/s | 44 kB 00:00 2023-10-07T07:55:16.736 INFO:teuthology.orchestra.run.smithi007.stdout:(96/139): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-10-07T07:55:16.755 INFO:teuthology.orchestra.run.smithi172.stdout:(66/139): keyutils-libs-devel-1.5.10-9.el8.x86_ 524 kB/s | 48 kB 00:00 2023-10-07T07:55:16.761 INFO:teuthology.orchestra.run.smithi007.stdout:(97/139): python3-cachetools-3.1.1-4.el8.noarch 395 kB/s | 33 kB 00:00 2023-10-07T07:55:16.787 INFO:teuthology.orchestra.run.smithi007.stdout:(98/139): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-10-07T07:55:16.812 INFO:teuthology.orchestra.run.smithi007.stdout:(99/139): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2023-10-07T07:55:16.822 INFO:teuthology.orchestra.run.smithi172.stdout:(67/139): fuse-2.9.7-17.el8.x86_64.rpm 368 kB/s | 83 kB 00:00 2023-10-07T07:55:16.881 INFO:teuthology.orchestra.run.smithi172.stdout:(68/139): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-10-07T07:55:16.887 INFO:teuthology.orchestra.run.smithi007.stdout:(100/139): python3-cherrypy-18.4.0-1.el8.noarch 3.8 MB/s | 384 kB 00:00 2023-10-07T07:55:16.904 INFO:teuthology.orchestra.run.smithi007.stdout:(101/139): python3-google-auth-1.1.1-10.el8.noa 883 kB/s | 81 kB 00:00 2023-10-07T07:55:16.938 INFO:teuthology.orchestra.run.smithi007.stdout:(102/139): python3-cheroot-8.5.2-1.el8.noarch.r 978 kB/s | 173 kB 00:00 2023-10-07T07:55:16.947 INFO:teuthology.orchestra.run.smithi172.stdout:(69/139): krb5-devel-1.18.2-22.el8.x86_64.rpm 2.9 MB/s | 561 kB 00:00 2023-10-07T07:55:16.963 INFO:teuthology.orchestra.run.smithi007.stdout:(103/139): python3-jaraco-6.2-6.el8.noarch.rpm 146 kB/s | 11 kB 00:00 2023-10-07T07:55:16.972 INFO:teuthology.orchestra.run.smithi172.stdout:(70/139): libcom_err-devel-1.45.6-5.el8.x86_64. 423 kB/s | 39 kB 00:00 2023-10-07T07:55:16.989 INFO:teuthology.orchestra.run.smithi007.stdout:(104/139): python3-jaraco-functools-2.0-4.el8.n 209 kB/s | 18 kB 00:00 2023-10-07T07:55:17.022 INFO:teuthology.orchestra.run.smithi007.stdout:(105/139): python3-logutils-0.3.5-11.el8.noarch 831 kB/s | 49 kB 00:00 2023-10-07T07:55:17.073 INFO:teuthology.orchestra.run.smithi007.stdout:(106/139): python3-kubernetes-11.0.0-6.el8.noar 8.7 MB/s | 1.2 MB 00:00 2023-10-07T07:55:17.081 INFO:teuthology.orchestra.run.smithi172.stdout:(71/139): libgfortran-8.5.0-20.el8.x86_64.rpm 5.8 MB/s | 645 kB 00:00 2023-10-07T07:55:17.098 INFO:teuthology.orchestra.run.smithi007.stdout:(107/139): python3-more-itertools-7.2.0-3.el8.n 539 kB/s | 59 kB 00:00 2023-10-07T07:55:17.106 INFO:teuthology.orchestra.run.smithi172.stdout:(72/139): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2023-10-07T07:55:17.140 INFO:teuthology.orchestra.run.smithi172.stdout:(73/139): librabbitmq-0.9.0-4.el8.x86_64.rpm 802 kB/s | 47 kB 00:00 2023-10-07T07:55:17.157 INFO:teuthology.orchestra.run.smithi007.stdout:(108/139): python3-natsort-7.1.1-2.el8.noarch.r 450 kB/s | 60 kB 00:00 2023-10-07T07:55:17.200 INFO:teuthology.orchestra.run.smithi172.stdout:(74/139): libselinux-devel-2.9-8.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-10-07T07:55:17.249 INFO:teuthology.orchestra.run.smithi172.stdout:(75/139): libsepol-devel-2.9-3.el8.x86_64.rpm 799 kB/s | 87 kB 00:00 2023-10-07T07:55:17.270 INFO:teuthology.orchestra.run.smithi007.stdout:(109/139): python3-portend-2.6-1.el8.noarch.rpm 91 kB/s | 16 kB 00:00 2023-10-07T07:55:17.274 INFO:teuthology.orchestra.run.smithi172.stdout:(76/139): libuuid-devel-2.32.1-43.el8.x86_64.rp 1.3 MB/s | 100 kB 00:00 2023-10-07T07:55:17.308 INFO:teuthology.orchestra.run.smithi172.stdout:(77/139): libverto-devel-0.3.2-2.el8.x86_64.rpm 310 kB/s | 18 kB 00:00 2023-10-07T07:55:17.333 INFO:teuthology.orchestra.run.smithi172.stdout:(78/139): libzstd-devel-1.4.4-1.el8.x86_64.rpm 751 kB/s | 44 kB 00:00 2023-10-07T07:55:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:(110/139): python3-repoze-lru-0.7-6.el8.noarch. 155 kB/s | 34 kB 00:00 2023-10-07T07:55:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:(111/139): python3-routes-2.4.1-12.el8.noarch.r 1.3 MB/s | 196 kB 00:00 2023-10-07T07:55:17.475 INFO:teuthology.orchestra.run.smithi172.stdout:(79/139): openssl-devel-1.1.1k-9.el8.x86_64.rpm 19 MB/s | 3.2 MB 00:00 2023-10-07T07:55:17.500 INFO:teuthology.orchestra.run.smithi172.stdout:(80/139): pcre2-devel-10.32-3.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2023-10-07T07:55:17.534 INFO:teuthology.orchestra.run.smithi172.stdout:(81/139): pcre2-utf16-10.32-3.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-10-07T07:55:17.557 INFO:teuthology.orchestra.run.smithi007.stdout:(112/139): python3-pecan-1.3.2-9.el8.noarch.rpm 583 kB/s | 283 kB 00:00 2023-10-07T07:55:17.576 INFO:teuthology.orchestra.run.smithi172.stdout:(82/139): pcre2-utf32-10.32-3.el8.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2023-10-07T07:55:17.583 INFO:teuthology.orchestra.run.smithi007.stdout:(113/139): python3-simplegeneric-0.8.1-17.el8.n 115 kB/s | 19 kB 00:00 2023-10-07T07:55:17.608 INFO:teuthology.orchestra.run.smithi007.stdout:(114/139): python3-rsa-4.9-2.el8.noarch.rpm 274 kB/s | 64 kB 00:00 2023-10-07T07:55:17.609 INFO:teuthology.orchestra.run.smithi172.stdout:(83/139): python3-cffi-1.11.5-6.el8.x86_64.rpm 3.2 MB/s | 249 kB 00:00 2023-10-07T07:55:17.633 INFO:teuthology.orchestra.run.smithi007.stdout:(115/139): python3-singledispatch-3.4.0.3-18.el 317 kB/s | 24 kB 00:00 2023-10-07T07:55:17.659 INFO:teuthology.orchestra.run.smithi007.stdout:(116/139): python3-tempora-1.14.1-5.el8.noarch. 387 kB/s | 29 kB 00:00 2023-10-07T07:55:17.668 INFO:teuthology.orchestra.run.smithi172.stdout:(84/139): python3-cryptography-3.2.1-6.el8.x86_ 6.8 MB/s | 641 kB 00:00 2023-10-07T07:55:17.693 INFO:teuthology.orchestra.run.smithi172.stdout:(85/139): python3-jwt-1.6.1-2.el8.noarch.rpm 510 kB/s | 43 kB 00:00 2023-10-07T07:55:17.706 INFO:teuthology.orchestra.run.smithi007.stdout:(117/139): python3-trustme-0.6.0-4.el8.noarch.r 281 kB/s | 27 kB 00:00 2023-10-07T07:55:17.727 INFO:teuthology.orchestra.run.smithi172.stdout:(86/139): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2023-10-07T07:55:17.734 INFO:teuthology.orchestra.run.smithi007.stdout:(118/139): python3-waitress-1.4.3-1.el8.noarch. 2.5 MB/s | 255 kB 00:00 2023-10-07T07:55:17.752 INFO:teuthology.orchestra.run.smithi172.stdout:(87/139): python3-pycparser-2.14-14.el8.noarch. 1.8 MB/s | 109 kB 00:00 2023-10-07T07:55:17.759 INFO:teuthology.orchestra.run.smithi007.stdout:(119/139): python3-webob-1.8.5-1.el8.1.noarch.r 2.4 MB/s | 251 kB 00:00 2023-10-07T07:55:17.785 INFO:teuthology.orchestra.run.smithi007.stdout:(120/139): python3-websocket-client-0.56.0-5.el 783 kB/s | 61 kB 00:00 2023-10-07T07:55:17.811 INFO:teuthology.orchestra.run.smithi172.stdout:(88/139): python3-requests-oauthlib-1.0.0-1.el8 510 kB/s | 43 kB 00:00 2023-10-07T07:55:17.818 INFO:teuthology.orchestra.run.smithi007.stdout:(121/139): python3-zc-lockfile-2.0-2.el8.noarch 394 kB/s | 23 kB 00:00 2023-10-07T07:55:17.844 INFO:teuthology.orchestra.run.smithi007.stdout:(122/139): python3-webtest-2.0.33-1.el8.noarch. 1.0 MB/s | 85 kB 00:00 2023-10-07T07:55:17.844 INFO:teuthology.orchestra.run.smithi172.stdout:(89/139): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 809 kB/s | 74 kB 00:00 2023-10-07T07:55:17.870 INFO:teuthology.orchestra.run.smithi172.stdout:(90/139): zlib-devel-1.2.11-21.el8.x86_64.rpm 993 kB/s | 58 kB 00:00 2023-10-07T07:55:17.961 INFO:teuthology.orchestra.run.smithi007.stdout:(123/139): thrift-0.13.0-2.el8.x86_64.rpm 9.9 MB/s | 1.7 MB 00:00 2023-10-07T07:55:18.028 INFO:teuthology.orchestra.run.smithi007.stdout:(124/139): NetworkManager-initscripts-updown-1. 2.1 MB/s | 143 kB 00:00 2023-10-07T07:55:18.086 INFO:teuthology.orchestra.run.smithi007.stdout:(125/139): elfutils-debuginfod-client-0.189-3.e 1.3 MB/s | 76 kB 00:00 2023-10-07T07:55:18.129 INFO:teuthology.orchestra.run.smithi172.stdout:(91/139): liboath-2.6.2-3.el8.x86_64.rpm 227 kB/s | 59 kB 00:00 2023-10-07T07:55:18.145 INFO:teuthology.orchestra.run.smithi007.stdout:(126/139): elfutils-libelf-0.189-3.el8.x86_64.r 3.9 MB/s | 232 kB 00:00 2023-10-07T07:55:18.212 INFO:teuthology.orchestra.run.smithi007.stdout:(127/139): elfutils-libs-0.189-3.el8.x86_64.rpm 4.4 MB/s | 303 kB 00:00 2023-10-07T07:55:18.212 INFO:teuthology.orchestra.run.smithi172.stdout:(92/139): gperftools-libs-2.7-9.el8.x86_64.rpm 834 kB/s | 306 kB 00:00 2023-10-07T07:55:18.246 INFO:teuthology.orchestra.run.smithi172.stdout:(93/139): libunwind-1.3.1-3.el8.x86_64.rpm 643 kB/s | 75 kB 00:00 2023-10-07T07:55:18.313 INFO:teuthology.orchestra.run.smithi172.stdout:(94/139): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-10-07T07:55:18.320 INFO:teuthology.orchestra.run.smithi007.stdout:(128/139): iproute-6.2.0-5.el8.x86_64.rpm 7.7 MB/s | 854 kB 00:00 2023-10-07T07:55:18.358 INFO:teuthology.orchestra.run.smithi172.stdout:(95/139): python3-asyncssh-2.7.0-2.el8.noarch.r 3.2 MB/s | 476 kB 00:00 2023-10-07T07:55:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:(129/139): libblkid-2.32.1-43.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-10-07T07:55:18.430 INFO:teuthology.orchestra.run.smithi172.stdout:(96/139): python3-cachetools-3.1.1-4.el8.noarch 457 kB/s | 33 kB 00:00 2023-10-07T07:55:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:(130/139): libfdisk-2.32.1-43.el8.x86_64.rpm 3.7 MB/s | 254 kB 00:00 2023-10-07T07:55:18.455 INFO:teuthology.orchestra.run.smithi172.stdout:(97/139): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-10-07T07:55:18.497 INFO:teuthology.orchestra.run.smithi172.stdout:(98/139): python3-certifi-2018.10.15-7.el8.noar 244 kB/s | 16 kB 00:00 2023-10-07T07:55:18.531 INFO:teuthology.orchestra.run.smithi172.stdout:(99/139): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-10-07T07:55:18.558 INFO:teuthology.orchestra.run.smithi007.stdout:(131/139): libmount-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 236 kB 00:00 2023-10-07T07:55:18.572 INFO:teuthology.orchestra.run.smithi172.stdout:(100/139): python3-cherrypy-18.4.0-1.el8.noarch 5.0 MB/s | 384 kB 00:00 2023-10-07T07:55:18.590 INFO:teuthology.orchestra.run.smithi172.stdout:(101/139): python3-google-auth-1.1.1-10.el8.noa 1.4 MB/s | 81 kB 00:00 2023-10-07T07:55:18.622 INFO:teuthology.orchestra.run.smithi007.stdout:(132/139): libsmartcols-2.32.1-43.el8.x86_64.rp 2.8 MB/s | 179 kB 00:00 2023-10-07T07:55:18.631 INFO:teuthology.orchestra.run.smithi172.stdout:(102/139): python3-jaraco-6.2-6.el8.noarch.rpm 189 kB/s | 11 kB 00:00 2023-10-07T07:55:18.649 INFO:teuthology.orchestra.run.smithi172.stdout:(103/139): python3-jaraco-functools-2.0-4.el8.n 299 kB/s | 18 kB 00:00 2023-10-07T07:55:18.681 INFO:teuthology.orchestra.run.smithi007.stdout:(133/139): libuuid-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 99 kB 00:00 2023-10-07T07:55:18.708 INFO:teuthology.orchestra.run.smithi172.stdout:(104/139): python3-logutils-0.3.5-11.el8.noarch 836 kB/s | 49 kB 00:00 2023-10-07T07:55:18.766 INFO:teuthology.orchestra.run.smithi172.stdout:(105/139): python3-kubernetes-11.0.0-6.el8.noar 8.7 MB/s | 1.2 MB 00:00 2023-10-07T07:55:18.791 INFO:teuthology.orchestra.run.smithi172.stdout:(106/139): python3-more-itertools-7.2.0-3.el8.n 704 kB/s | 59 kB 00:00 2023-10-07T07:55:18.834 INFO:teuthology.orchestra.run.smithi172.stdout:(107/139): python3-natsort-7.1.1-2.el8.noarch.r 900 kB/s | 60 kB 00:00 2023-10-07T07:55:18.853 INFO:teuthology.orchestra.run.smithi007.stdout:(134/139): util-linux-2.32.1-43.el8.x86_64.rpm 14 MB/s | 2.5 MB 00:00 2023-10-07T07:55:18.867 INFO:teuthology.orchestra.run.smithi172.stdout:(108/139): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-10-07T07:55:18.900 INFO:teuthology.orchestra.run.smithi172.stdout:(109/139): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-10-07T07:55:18.914 INFO:teuthology.orchestra.run.smithi007.stdout:(135/139): util-linux-user-2.32.1-43.el8.x86_64 1.7 MB/s | 103 kB 00:00 2023-10-07T07:55:18.990 INFO:teuthology.orchestra.run.smithi007.stdout:(136/139): librados2-17.2.6-1320.g08cac018.el8. 3.0 MB/s | 3.5 MB 00:01 2023-10-07T07:55:19.049 INFO:teuthology.orchestra.run.smithi007.stdout:(137/139): librbd1-17.2.6-1320.g08cac018.el8.x8 3.0 MB/s | 3.6 MB 00:01 2023-10-07T07:55:19.074 INFO:teuthology.orchestra.run.smithi007.stdout:(138/139): xfsdump-3.1.8-6.el8.x86_64.rpm 2.1 MB/s | 334 kB 00:00 2023-10-07T07:55:19.324 INFO:teuthology.orchestra.run.smithi007.stdout:(139/139): xfsprogs-5.0.0-12.el8.x86_64.rpm 3.2 MB/s | 1.1 MB 00:00 2023-10-07T07:55:19.327 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2023-10-07T07:55:19.327 INFO:teuthology.orchestra.run.smithi007.stdout:Total 13 MB/s | 210 MB 00:16 2023-10-07T07:55:19.351 INFO:teuthology.orchestra.run.smithi172.stdout:(110/139): ceph-test-17.2.6-1320.g08cac018.el8. 5.3 MB/s | 52 MB 00:09 2023-10-07T07:55:19.376 INFO:teuthology.orchestra.run.smithi172.stdout:(111/139): python3-repoze-lru-0.7-6.el8.noarch. 66 kB/s | 34 kB 00:00 2023-10-07T07:55:19.401 INFO:teuthology.orchestra.run.smithi172.stdout:(112/139): python3-routes-2.4.1-12.el8.noarch.r 393 kB/s | 196 kB 00:00 2023-10-07T07:55:19.443 INFO:teuthology.orchestra.run.smithi172.stdout:(113/139): python3-simplegeneric-0.8.1-17.el8.n 279 kB/s | 19 kB 00:00 2023-10-07T07:55:19.468 INFO:teuthology.orchestra.run.smithi172.stdout:(114/139): python3-singledispatch-3.4.0.3-18.el 358 kB/s | 24 kB 00:00 2023-10-07T07:55:19.527 INFO:teuthology.orchestra.run.smithi172.stdout:(115/139): python3-tempora-1.14.1-5.el8.noarch. 350 kB/s | 29 kB 00:00 2023-10-07T07:55:19.552 INFO:teuthology.orchestra.run.smithi172.stdout:(116/139): python3-trustme-0.6.0-4.el8.noarch.r 329 kB/s | 27 kB 00:00 2023-10-07T07:55:19.578 INFO:teuthology.orchestra.run.smithi172.stdout:(117/139): python3-rsa-4.9-2.el8.noarch.rpm 284 kB/s | 64 kB 00:00 2023-10-07T07:55:19.628 INFO:teuthology.orchestra.run.smithi172.stdout:(118/139): python3-webob-1.8.5-1.el8.1.noarch.r 3.3 MB/s | 251 kB 00:00 2023-10-07T07:55:19.645 INFO:teuthology.orchestra.run.smithi172.stdout:(119/139): python3-websocket-client-0.56.0-5.el 917 kB/s | 61 kB 00:00 2023-10-07T07:55:19.670 INFO:teuthology.orchestra.run.smithi172.stdout:(120/139): python3-waitress-1.4.3-1.el8.noarch. 1.7 MB/s | 255 kB 00:00 2023-10-07T07:55:19.695 INFO:teuthology.orchestra.run.smithi172.stdout:(121/139): python3-webtest-2.0.33-1.el8.noarch. 1.2 MB/s | 85 kB 00:00 2023-10-07T07:55:19.721 INFO:teuthology.orchestra.run.smithi172.stdout:(122/139): python3-zc-lockfile-2.0-2.el8.noarch 304 kB/s | 23 kB 00:00 2023-10-07T07:55:19.854 INFO:teuthology.orchestra.run.smithi172.stdout:(123/139): thrift-0.13.0-2.el8.x86_64.rpm 9.4 MB/s | 1.7 MB 00:00 2023-10-07T07:55:20.086 INFO:teuthology.orchestra.run.smithi172.stdout:(124/139): NetworkManager-initscripts-updown-1. 619 kB/s | 143 kB 00:00 2023-10-07T07:55:20.155 INFO:teuthology.orchestra.run.smithi172.stdout:(125/139): elfutils-debuginfod-client-0.189-3.e 1.1 MB/s | 76 kB 00:00 2023-10-07T07:55:20.260 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2023-10-07T07:55:20.263 INFO:teuthology.orchestra.run.smithi172.stdout:(126/139): librados2-17.2.6-1320.g08cac018.el8. 6.1 MB/s | 3.5 MB 00:00 2023-10-07T07:55:20.289 INFO:teuthology.orchestra.run.smithi172.stdout:(127/139): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2023-10-07T07:55:20.339 INFO:teuthology.orchestra.run.smithi172.stdout:(128/139): librbd1-17.2.6-1320.g08cac018.el8.x8 5.8 MB/s | 3.6 MB 00:00 2023-10-07T07:55:20.431 INFO:teuthology.orchestra.run.smithi172.stdout:(129/139): iproute-6.2.0-5.el8.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2023-10-07T07:55:20.489 INFO:teuthology.orchestra.run.smithi172.stdout:(130/139): libfdisk-2.32.1-43.el8.x86_64.rpm 4.3 MB/s | 254 kB 00:00 2023-10-07T07:55:20.548 INFO:teuthology.orchestra.run.smithi172.stdout:(131/139): elfutils-libs-0.189-3.el8.x86_64.rpm 1.0 MB/s | 303 kB 00:00 2023-10-07T07:55:20.573 INFO:teuthology.orchestra.run.smithi172.stdout:(132/139): libblkid-2.32.1-43.el8.x86_64.rpm 943 kB/s | 220 kB 00:00 2023-10-07T07:55:20.599 INFO:teuthology.orchestra.run.smithi172.stdout:(133/139): libmount-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 236 kB 00:00 2023-10-07T07:55:20.624 INFO:teuthology.orchestra.run.smithi172.stdout:(134/139): libsmartcols-2.32.1-43.el8.x86_64.rp 2.3 MB/s | 179 kB 00:00 2023-10-07T07:55:20.681 INFO:teuthology.orchestra.run.smithi172.stdout:(135/139): libuuid-2.32.1-43.el8.x86_64.rpm 922 kB/s | 99 kB 00:00 2023-10-07T07:55:20.708 INFO:teuthology.orchestra.run.smithi172.stdout:(136/139): util-linux-user-2.32.1-43.el8.x86_64 1.2 MB/s | 103 kB 00:00 2023-10-07T07:55:20.742 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2023-10-07T07:55:20.742 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2023-10-07T07:55:20.775 INFO:teuthology.orchestra.run.smithi172.stdout:(137/139): util-linux-2.32.1-43.el8.x86_64.rpm 14 MB/s | 2.5 MB 00:00 2023-10-07T07:55:20.800 INFO:teuthology.orchestra.run.smithi172.stdout:(138/139): xfsdump-3.1.8-6.el8.x86_64.rpm 2.7 MB/s | 334 kB 00:00 2023-10-07T07:55:20.892 INFO:teuthology.orchestra.run.smithi172.stdout:(139/139): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.9 MB/s | 1.1 MB 00:00 2023-10-07T07:55:20.894 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2023-10-07T07:55:20.895 INFO:teuthology.orchestra.run.smithi172.stdout:Total 10 MB/s | 210 MB 00:20 2023-10-07T07:55:21.750 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2023-10-07T07:55:22.176 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2023-10-07T07:55:22.177 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2023-10-07T07:55:22.266 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2023-10-07T07:55:22.269 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2023-10-07T07:55:23.623 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2023-10-07T07:55:23.626 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2023-10-07T07:55:28.134 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2023-10-07T07:55:28.187 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-10-07T07:55:28.400 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-07T07:55:28.473 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-07T07:55:28.675 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-07T07:55:28.756 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-07T07:55:28.849 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2023-10-07T07:55:28.903 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-10-07T07:55:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/155 2023-10-07T07:55:29.053 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-07T07:55:29.134 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-07T07:55:29.311 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-07T07:55:29.341 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 4/155 2023-10-07T07:55:29.388 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-07T07:55:29.564 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/155 2023-10-07T07:55:29.598 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/155 2023-10-07T07:55:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/155 2023-10-07T07:55:29.923 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 4/155 2023-10-07T07:55:30.033 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-07T07:55:30.132 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/155 2023-10-07T07:55:30.324 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/155 2023-10-07T07:55:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-07T07:55:30.536 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-07T07:55:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 8/155 2023-10-07T07:55:30.919 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-07T07:55:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 8/155 2023-10-07T07:55:31.135 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 8/155 2023-10-07T07:55:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 9/155 2023-10-07T07:55:31.454 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 9/155 2023-10-07T07:55:31.529 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 8/155 2023-10-07T07:55:31.658 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 10/155 2023-10-07T07:55:31.727 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 9/155 2023-10-07T07:55:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 10/155 2023-10-07T07:55:31.854 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 9/155 2023-10-07T07:55:32.011 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 11/155 2023-10-07T07:55:32.027 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 10/155 2023-10-07T07:55:32.109 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 10/155 2023-10-07T07:55:32.191 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-07T07:55:32.342 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 11/155 2023-10-07T07:55:32.502 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-07T07:55:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-07T07:55:32.983 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 13/155 2023-10-07T07:55:33.074 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-07T07:55:33.239 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 13/155 2023-10-07T07:55:33.265 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 13/155 2023-10-07T07:55:33.486 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 13/155 2023-10-07T07:55:33.542 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-07T07:55:33.660 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-07T07:55:33.766 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-07T07:55:33.838 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-07T07:55:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/155 2023-10-07T07:55:34.056 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/155 2023-10-07T07:55:34.066 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/155 2023-10-07T07:55:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 17/155 2023-10-07T07:55:34.263 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/155 2023-10-07T07:55:34.323 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 17/155 2023-10-07T07:55:34.393 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 17/155 2023-10-07T07:55:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/155 2023-10-07T07:55:34.499 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 17/155 2023-10-07T07:55:34.610 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/155 2023-10-07T07:55:34.643 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/155 2023-10-07T07:55:34.746 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/155 2023-10-07T07:55:34.807 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/155 2023-10-07T07:55:34.916 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/155 2023-10-07T07:55:34.971 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/155 2023-10-07T07:55:35.085 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/155 2023-10-07T07:55:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-07T07:55:35.235 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-07T07:55:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-07T07:55:35.390 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-07T07:55:35.457 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-07T07:55:35.526 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-07T07:55:36.106 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-07T07:55:36.122 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-07T07:55:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/155 2023-10-07T07:55:36.376 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/155 2023-10-07T07:55:36.993 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/155 2023-10-07T07:55:37.027 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/155 2023-10-07T07:55:37.143 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-07T07:55:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-07T07:55:37.848 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-07T07:55:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-07T07:55:38.211 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/155 2023-10-07T07:55:38.317 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/155 2023-10-07T07:55:38.390 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/155 2023-10-07T07:55:38.550 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/155 2023-10-07T07:55:38.580 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/155 2023-10-07T07:55:38.705 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/155 2023-10-07T07:55:38.769 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/155 2023-10-07T07:55:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/155 2023-10-07T07:55:38.949 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 29/155 2023-10-07T07:55:39.128 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 30/155 2023-10-07T07:55:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 29/155 2023-10-07T07:55:39.311 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 30/155 2023-10-07T07:55:39.329 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 31/155 2023-10-07T07:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 31/155 2023-10-07T07:55:40.293 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 32/155 2023-10-07T07:55:40.635 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 32/155 2023-10-07T07:55:40.660 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/155 2023-10-07T07:55:40.972 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/155 2023-10-07T07:55:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/155 2023-10-07T07:55:41.167 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-07T07:55:41.451 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/155 2023-10-07T07:55:41.656 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-07T07:55:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-07T07:55:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 36/155 2023-10-07T07:55:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/155 2023-10-07T07:55:43.688 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-07T07:55:44.012 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/155 2023-10-07T07:55:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 36/155 2023-10-07T07:55:44.202 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 39/155 2023-10-07T07:55:44.242 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/155 2023-10-07T07:55:44.389 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 40/155 2023-10-07T07:55:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/155 2023-10-07T07:55:44.626 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 41/155 2023-10-07T07:55:44.679 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 39/155 2023-10-07T07:55:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 42/155 2023-10-07T07:55:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 40/155 2023-10-07T07:55:44.992 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 43/155 2023-10-07T07:55:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 41/155 2023-10-07T07:55:45.191 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-07T07:55:45.264 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-07T07:55:45.361 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 42/155 2023-10-07T07:55:45.437 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/155 2023-10-07T07:55:45.549 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/155 2023-10-07T07:55:45.550 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 43/155 2023-10-07T07:55:45.687 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 46/155 2023-10-07T07:55:45.772 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-07T07:55:45.841 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-07T07:55:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-07T07:55:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/155 2023-10-07T07:55:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/155 2023-10-07T07:55:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 46/155 2023-10-07T07:55:46.374 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-07T07:55:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-07T07:55:46.617 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-07T07:55:46.631 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-07T07:55:46.810 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/155 2023-10-07T07:55:47.075 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/155 2023-10-07T07:55:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-07T07:55:47.297 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/155 2023-10-07T07:55:47.373 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-07T07:55:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-07T07:55:47.453 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/155 2023-10-07T07:55:47.586 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/155 2023-10-07T07:55:47.601 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/155 2023-10-07T07:55:47.773 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/155 2023-10-07T07:55:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/155 2023-10-07T07:55:47.985 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/155 2023-10-07T07:55:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/155 2023-10-07T07:55:48.159 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/155 2023-10-07T07:55:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/155 2023-10-07T07:55:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/155 2023-10-07T07:55:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/155 2023-10-07T07:55:48.471 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/155 2023-10-07T07:55:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/155 2023-10-07T07:55:48.644 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/155 2023-10-07T07:55:48.797 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/155 2023-10-07T07:55:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/155 2023-10-07T07:55:48.982 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/155 2023-10-07T07:55:49.163 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/155 2023-10-07T07:55:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/155 2023-10-07T07:55:49.359 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/155 2023-10-07T07:55:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/155 2023-10-07T07:55:49.623 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/155 2023-10-07T07:55:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/155 2023-10-07T07:55:49.819 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/155 2023-10-07T07:55:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/155 2023-10-07T07:55:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/155 2023-10-07T07:55:50.148 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/155 2023-10-07T07:55:50.375 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/155 2023-10-07T07:55:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/155 2023-10-07T07:55:50.574 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/155 2023-10-07T07:55:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/155 2023-10-07T07:55:50.804 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/155 2023-10-07T07:55:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/155 2023-10-07T07:55:51.044 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 70/155 2023-10-07T07:55:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/155 2023-10-07T07:55:51.296 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/155 2023-10-07T07:55:51.440 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 71/155 2023-10-07T07:55:51.669 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/155 2023-10-07T07:55:51.710 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 72/155 2023-10-07T07:55:51.898 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/155 2023-10-07T07:55:51.942 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 73/155 2023-10-07T07:55:52.135 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/155 2023-10-07T07:55:52.162 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/155 2023-10-07T07:55:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/155 2023-10-07T07:55:52.386 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/155 2023-10-07T07:55:52.627 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 70/155 2023-10-07T07:55:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 71/155 2023-10-07T07:55:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 72/155 2023-10-07T07:55:53.440 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/155 2023-10-07T07:55:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 73/155 2023-10-07T07:55:53.809 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 77/155 2023-10-07T07:55:53.825 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/155 2023-10-07T07:55:54.039 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 78/155 2023-10-07T07:55:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/155 2023-10-07T07:55:54.333 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 79/155 2023-10-07T07:55:54.623 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/155 2023-10-07T07:55:54.858 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 81/155 2023-10-07T07:55:55.084 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/155 2023-10-07T07:55:55.268 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-07T07:55:55.405 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/155 2023-10-07T07:55:55.591 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 84/155 2023-10-07T07:55:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 77/155 2023-10-07T07:55:55.811 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 85/155 2023-10-07T07:55:55.967 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 86/155 2023-10-07T07:55:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 78/155 2023-10-07T07:55:56.146 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 87/155 2023-10-07T07:55:56.192 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 79/155 2023-10-07T07:55:56.457 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 88/155 2023-10-07T07:55:56.513 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/155 2023-10-07T07:55:56.693 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 89/155 2023-10-07T07:55:56.741 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 81/155 2023-10-07T07:55:56.835 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 90/155 2023-10-07T07:55:57.018 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/155 2023-10-07T07:55:57.037 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 91/155 2023-10-07T07:55:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-07T07:55:57.274 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 92/155 2023-10-07T07:55:57.482 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.7-17.el8.x86_64 93/155 2023-10-07T07:55:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 84/155 2023-10-07T07:55:57.759 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 94/155 2023-10-07T07:55:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 85/155 2023-10-07T07:55:57.947 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 95/155 2023-10-07T07:55:58.056 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 86/155 2023-10-07T07:55:58.065 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/155 2023-10-07T07:55:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 87/155 2023-10-07T07:55:58.621 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 88/155 2023-10-07T07:55:58.884 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 89/155 2023-10-07T07:55:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 90/155 2023-10-07T07:55:59.288 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 91/155 2023-10-07T07:55:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 92/155 2023-10-07T07:55:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.7-17.el8.x86_64 93/155 2023-10-07T07:56:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 94/155 2023-10-07T07:56:00.322 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 95/155 2023-10-07T07:56:00.462 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/155 2023-10-07T07:56:00.693 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 97/155 2023-10-07T07:56:00.897 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 97/155 2023-10-07T07:56:01.458 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 97/155 2023-10-07T07:56:01.751 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 98/155 2023-10-07T07:56:01.858 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 98/155 2023-10-07T07:56:02.022 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 99/155 2023-10-07T07:56:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 97/155 2023-10-07T07:56:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 97/155 2023-10-07T07:56:04.222 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 97/155 2023-10-07T07:56:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 98/155 2023-10-07T07:56:04.680 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 98/155 2023-10-07T07:56:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 99/155 2023-10-07T07:56:20.310 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 99/155 2023-10-07T07:56:20.310 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2023-10-07T07:56:20.310 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2023-10-07T07:56:20.310 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /dev 2023-10-07T07:56:20.310 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /run 2023-10-07T07:56:20.311 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2023-10-07T07:56:20.311 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2023-10-07T07:56:20.311 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2023-10-07T07:56:20.311 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2023-10-07T07:56:20.311 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /dev 2023-10-07T07:56:20.311 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:20.685 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 100/155 2023-10-07T07:56:22.164 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 100/155 2023-10-07T07:56:22.389 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 101/155 2023-10-07T07:56:22.611 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 101/155 2023-10-07T07:56:22.818 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 102/155 2023-10-07T07:56:23.209 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 102/155 2023-10-07T07:56:23.427 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 103/155 2023-10-07T07:56:23.451 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 103/155 2023-10-07T07:56:23.625 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/155 2023-10-07T07:56:23.804 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/155 2023-10-07T07:56:24.018 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-07T07:56:24.106 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-07T07:56:24.379 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : jq-1.6-7.el8.x86_64 107/155 2023-10-07T07:56:24.618 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.1-8.el8.noarch 108/155 2023-10-07T07:56:25.145 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 109/155 2023-10-07T07:56:25.324 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 109/155 2023-10-07T07:56:25.432 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 110/155 2023-10-07T07:56:25.596 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 111/155 2023-10-07T07:56:25.826 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 112/155 2023-10-07T07:56:26.036 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 112/155 2023-10-07T07:56:26.263 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 113/155 2023-10-07T07:56:26.525 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 99/155 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /run 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2023-10-07T07:56:26.526 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 100/155 2023-10-07T07:56:27.229 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 113/155 2023-10-07T07:56:27.426 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 114/155 2023-10-07T07:56:27.518 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 114/155 2023-10-07T07:56:27.892 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 115/155 2023-10-07T07:56:28.121 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 116/155 2023-10-07T07:56:28.158 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 116/155 2023-10-07T07:56:28.334 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 117/155 2023-10-07T07:56:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 117/155 2023-10-07T07:56:28.536 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 118/155 2023-10-07T07:56:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 100/155 2023-10-07T07:56:28.709 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/155 2023-10-07T07:56:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 101/155 2023-10-07T07:56:29.036 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 101/155 2023-10-07T07:56:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 102/155 2023-10-07T07:56:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 102/155 2023-10-07T07:56:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 103/155 2023-10-07T07:56:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 103/155 2023-10-07T07:56:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/155 2023-10-07T07:56:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/155 2023-10-07T07:56:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-07T07:56:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-07T07:56:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : jq-1.6-7.el8.x86_64 107/155 2023-10-07T07:56:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.1-8.el8.noarch 108/155 2023-10-07T07:56:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 109/155 2023-10-07T07:56:31.907 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 109/155 2023-10-07T07:56:32.081 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 110/155 2023-10-07T07:56:32.249 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 111/155 2023-10-07T07:56:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 112/155 2023-10-07T07:56:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 112/155 2023-10-07T07:56:32.920 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 113/155 2023-10-07T07:56:33.642 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/155 2023-10-07T07:56:33.993 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 113/155 2023-10-07T07:56:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 114/155 2023-10-07T07:56:34.296 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 114/155 2023-10-07T07:56:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 115/155 2023-10-07T07:56:34.946 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 116/155 2023-10-07T07:56:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 116/155 2023-10-07T07:56:35.189 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 117/155 2023-10-07T07:56:35.204 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 117/155 2023-10-07T07:56:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 118/155 2023-10-07T07:56:35.419 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 120/155 2023-10-07T07:56:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/155 2023-10-07T07:56:35.603 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 121/155 2023-10-07T07:56:35.693 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 121/155 2023-10-07T07:56:35.858 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 122/155 2023-10-07T07:56:35.900 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 122/155 2023-10-07T07:56:36.132 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 123/155 2023-10-07T07:56:36.479 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 123/155 2023-10-07T07:56:36.674 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 124/155 2023-10-07T07:56:36.769 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 124/155 2023-10-07T07:56:37.637 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 125/155 2023-10-07T07:56:37.987 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 126/155 2023-10-07T07:56:38.146 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/155 2023-10-07T07:56:38.360 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 128/155 2023-10-07T07:56:38.519 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 129/155 2023-10-07T07:56:38.734 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 130/155 2023-10-07T07:56:38.978 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 131/155 2023-10-07T07:56:39.174 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 132/155 2023-10-07T07:56:39.407 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 133/155 2023-10-07T07:56:39.790 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 134/155 2023-10-07T07:56:40.084 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 135/155 2023-10-07T07:56:40.249 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-07T07:56:40.265 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-07T07:56:40.446 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 137/155 2023-10-07T07:56:40.673 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.1-9.el8.x86_64 138/155 2023-10-07T07:56:40.771 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 138/155 2023-10-07T07:56:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/155 2023-10-07T07:56:40.990 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.0.4-10.el8.x86_64 139/155 2023-10-07T07:56:41.003 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 139/155 2023-10-07T07:56:41.225 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 140/155 2023-10-07T07:56:41.379 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 141/155 2023-10-07T07:56:41.539 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 142/155 2023-10-07T07:56:41.540 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-07T07:56:41.785 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-07T07:56:41.785 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-07T07:56:42.000 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-07T07:56:42.006 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-07T07:56:42.118 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-07T07:56:42.119 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-07T07:56:42.413 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-07T07:56:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 147/155 2023-10-07T07:56:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 148/155 2023-10-07T07:56:42.852 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 120/155 2023-10-07T07:56:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-07T07:56:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-07T07:56:43.069 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 121/155 2023-10-07T07:56:43.147 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 121/155 2023-10-07T07:56:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-07T07:56:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 151/155 2023-10-07T07:56:43.314 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 122/155 2023-10-07T07:56:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 151/155 2023-10-07T07:56:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-07T07:56:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 122/155 2023-10-07T07:56:43.578 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 123/155 2023-10-07T07:56:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-07T07:56:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-07T07:56:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-07T07:56:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-07T07:56:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 123/155 2023-10-07T07:56:43.984 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-07T07:56:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 124/155 2023-10-07T07:56:44.191 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-07T07:56:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 124/155 2023-10-07T07:56:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-07T07:56:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/155 2023-10-07T07:56:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/155 2023-10-07T07:56:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/155 2023-10-07T07:56:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/155 2023-10-07T07:56:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/155 2023-10-07T07:56:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : jq-1.6-7.el8.x86_64 42/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 43/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 44/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 45/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 46/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 47/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 48/155 2023-10-07T07:56:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 49/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 50/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 51/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 52/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 53/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 54/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 55/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 56/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 57/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 58/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 59/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 60/155 2023-10-07T07:56:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 61/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 62/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 63/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 64/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 65/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 66/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 67/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 68/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 69/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 70/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 71/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 72/155 2023-10-07T07:56:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 73/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 74/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 75/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 76/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 77/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 78/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 80/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 81/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 82/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 84/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 85/155 2023-10-07T07:56:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 86/155 2023-10-07T07:56:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 87/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 88/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 89/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 90/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 91/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 92/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 95/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/155 2023-10-07T07:56:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 102/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 103/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 104/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 105/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 106/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 107/155 2023-10-07T07:56:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 108/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 111/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 112/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 113/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 114/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 115/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 116/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 117/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 118/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 119/155 2023-10-07T07:56:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 120/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 121/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 122/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 123/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 124/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 126/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 128/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 130/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 131/155 2023-10-07T07:56:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 132/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 133/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 134/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 135/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 136/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 137/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 138/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 139/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 140/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 141/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/155 2023-10-07T07:56:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 143/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 145/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 147/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 151/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/155 2023-10-07T07:56:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 154/155 2023-10-07T07:56:45.287 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 125/155 2023-10-07T07:56:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 126/155 2023-10-07T07:56:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/155 2023-10-07T07:56:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 128/155 2023-10-07T07:56:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 129/155 2023-10-07T07:56:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 130/155 2023-10-07T07:56:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 131/155 2023-10-07T07:56:46.916 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 155/155 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-07T07:56:46.917 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.1-8.el8.noarch 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.0.4-10.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.918 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.919 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.1-9.el8.x86_64 2023-10-07T07:56:46.920 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: jq-1.6-7.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-07T07:56:46.921 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-07T07:56:46.922 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-07T07:56:46.923 INFO:teuthology.orchestra.run.smithi172.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-07T07:56:46.924 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-07T07:56:46.925 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-2.6-1.el8.noarch 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.926 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-07T07:56:46.927 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-10-07T07:56:46.928 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-07T07:56:46.929 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-07T07:56:46.929 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:46.929 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2023-10-07T07:56:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 132/155 2023-10-07T07:56:47.168 DEBUG:teuthology.parallel:result is None 2023-10-07T07:56:47.230 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 133/155 2023-10-07T07:56:47.618 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 134/155 2023-10-07T07:56:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 135/155 2023-10-07T07:56:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-07T07:56:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-07T07:56:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 137/155 2023-10-07T07:56:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.1-9.el8.x86_64 138/155 2023-10-07T07:56:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 138/155 2023-10-07T07:56:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.0.4-10.el8.x86_64 139/155 2023-10-07T07:56:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 139/155 2023-10-07T07:56:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 140/155 2023-10-07T07:56:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 141/155 2023-10-07T07:56:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 142/155 2023-10-07T07:56:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-07T07:56:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-07T07:56:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-07T07:56:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-07T07:56:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-07T07:56:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-07T07:56:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-07T07:56:50.352 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-07T07:56:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 147/155 2023-10-07T07:56:50.731 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 148/155 2023-10-07T07:56:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-07T07:56:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-07T07:56:51.181 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-07T07:56:51.181 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 151/155 2023-10-07T07:56:51.414 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 151/155 2023-10-07T07:56:51.414 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-07T07:56:51.647 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-07T07:56:51.648 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-07T07:56:51.839 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-07T07:56:51.839 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-07T07:56:52.064 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-07T07:56:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-07T07:56:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-07T07:56:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/155 2023-10-07T07:56:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/155 2023-10-07T07:56:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/155 2023-10-07T07:56:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/155 2023-10-07T07:56:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/155 2023-10-07T07:56:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/155 2023-10-07T07:56:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/155 2023-10-07T07:56:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/155 2023-10-07T07:56:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/155 2023-10-07T07:56:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/155 2023-10-07T07:56:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/155 2023-10-07T07:56:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/155 2023-10-07T07:56:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/155 2023-10-07T07:56:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/155 2023-10-07T07:56:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/155 2023-10-07T07:56:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/155 2023-10-07T07:56:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/155 2023-10-07T07:56:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/155 2023-10-07T07:56:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/155 2023-10-07T07:56:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/155 2023-10-07T07:56:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/155 2023-10-07T07:56:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/155 2023-10-07T07:56:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/155 2023-10-07T07:56:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/155 2023-10-07T07:56:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/155 2023-10-07T07:56:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : jq-1.6-7.el8.x86_64 42/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 43/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 44/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 45/155 2023-10-07T07:56:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 46/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 47/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 48/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 49/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 50/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 51/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 52/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 53/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 54/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 55/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 56/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 57/155 2023-10-07T07:56:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 58/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 59/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 60/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 61/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 62/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 63/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 64/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 65/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 66/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 67/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 68/155 2023-10-07T07:56:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 69/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 70/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 71/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 72/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 73/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 74/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 75/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 76/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 77/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 78/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 80/155 2023-10-07T07:56:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 81/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 82/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 84/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 85/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 86/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 87/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 88/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 89/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 90/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 91/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 92/155 2023-10-07T07:56:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 95/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 102/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 103/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 104/155 2023-10-07T07:56:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 105/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 106/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 107/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 108/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 111/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 112/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 113/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 114/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 115/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 116/155 2023-10-07T07:56:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 117/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 118/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 119/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 120/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 121/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 122/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 123/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 124/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 126/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 128/155 2023-10-07T07:56:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 130/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 131/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 132/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 133/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 134/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 135/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 136/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 137/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 138/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 139/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 140/155 2023-10-07T07:56:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 141/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 143/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 145/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 147/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 151/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/155 2023-10-07T07:56:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/155 2023-10-07T07:56:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 154/155 2023-10-07T07:56:55.372 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 155/155 2023-10-07T07:56:55.372 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-07T07:56:55.373 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.1-8.el8.noarch 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.0.4-10.el8.x86_64 2023-10-07T07:56:55.374 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.375 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.376 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.1-9.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: jq-1.6-7.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-07T07:56:55.378 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-07T07:56:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.380 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-07T07:56:55.381 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-07T07:56:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-2.6-1.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-07T07:56:55.383 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-07T07:56:55.384 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-07T07:56:55.385 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-07T07:56:55.385 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.385 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.385 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T07:56:55.385 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-07T07:56:55.385 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-07T07:56:55.385 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-10-07T07:56:55.385 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-07T07:56:55.385 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-07T07:56:55.386 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:55.386 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2023-10-07T07:56:55.599 DEBUG:teuthology.parallel:result is None 2023-10-07T07:56:55.600 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T07:56:55.948 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-07T07:56:55.978 INFO:teuthology.orchestra.run.smithi007.stdout:17.2.6-1320.g08cac018.el8 2023-10-07T07:56:55.979 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-07T07:56:55.979 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-07T07:56:55.979 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T07:56:56.210 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-07T07:56:56.240 INFO:teuthology.orchestra.run.smithi172.stdout:17.2.6-1320.g08cac018.el8 2023-10-07T07:56:56.240 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-07T07:56:56.240 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-07T07:56:56.240 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-07T07:56:56.241 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:56:56.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-07T07:56:56.281 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:56:56.281 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-07T07:56:56.317 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-07T07:56:56.317 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:56:56.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2023-10-07T07:56:56.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-07T07:56:56.428 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:56:56.428 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2023-10-07T07:56:56.461 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-07T07:56:56.531 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-07T07:56:56.532 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:56:56.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-07T07:56:56.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-07T07:56:56.641 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:56:56.641 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-07T07:56:56.672 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-07T07:56:56.743 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-07T07:56:56.743 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:56:56.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2023-10-07T07:56:56.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-07T07:56:56.851 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:56:56.851 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2023-10-07T07:56:56.880 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-07T07:56:56.949 INFO:teuthology.run_tasks:Running task ceph... 2023-10-07T07:56:57.013 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-07T07:56:57.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2023-10-07T07:56:57.015 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /var/log/ceph 2023-10-07T07:56:57.047 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-07T07:56:57.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-07T07:56:57.090 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-07T07:56:57.121 INFO:tasks.ceph:Creating extra log directories... 2023-10-07T07:56:57.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-07T07:56:57.164 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-07T07:56:57.204 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-07T07:56:57.204 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-07T07:56:57.204 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-07T07:56:57.204 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:56:57.204 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-07T07:56:57.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-07T07:56:57.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-07T07:56:57.424 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:56:57.424 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-07T07:56:57.442 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-07T07:56:57.557 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-07T07:56:57.590 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-07T07:56:57.591 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-07T07:56:57.591 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223', 'branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7416223', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-06-0949-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4726, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDvzTKOQgOGweamAbVUNfrC8pj/dSyjZ9V0tg0+C7IOj4n90Q1to5OmKeP+Jy9J1prx3MYhGj8ZNLJlE75U6p9w=', 'smithi172.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD/VRA/QSRFLcfZ0A2n4Zd6jOOh9njj/CIoNcl1+A3795u1aCYQmZKDV32dGZGcnYvugk/tG7M6tUZ9JXtW67cA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-07_00:40:05', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-07T07:56:57.591 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-07T07:56:57.594 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-07T07:56:57.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2023-10-07T07:56:57.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/run/ceph 2023-10-07T07:56:57.711 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:56:57.711 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2023-10-07T07:56:57.753 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-07T07:56:57.754 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2023-10-07T07:56:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-07T07:56:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T07:56:57.818 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 396038 Links: 1 2023-10-07T07:56:57.818 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T07:56:57.818 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T07:56:57.818 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-10-07 07:56:52.524059558 +0000 2023-10-07T07:56:57.818 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-10-07 07:53:46.226797531 +0000 2023-10-07T07:56:57.818 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-10-07 07:53:46.226797531 +0000 2023-10-07T07:56:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2023-10-07T07:56:57.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-07T07:56:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-10-07T07:56:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-10-07T07:56:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000130151 s, 3.9 MB/s 2023-10-07T07:56:57.893 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-07T07:56:57.958 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2023-10-07T07:56:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-07T07:56:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T07:56:58.093 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 398650 Links: 1 2023-10-07T07:56:58.093 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T07:56:58.093 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T07:56:58.093 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-10-07 07:56:52.524059558 +0000 2023-10-07T07:56:58.093 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-10-07 07:53:46.771789453 +0000 2023-10-07T07:56:58.093 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-10-07 07:53:46.771789453 +0000 2023-10-07T07:56:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2023-10-07T07:56:58.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-07T07:56:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-10-07T07:56:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-10-07T07:56:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000117946 s, 4.3 MB/s 2023-10-07T07:56:58.128 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-07T07:56:58.192 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2023-10-07T07:56:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-07T07:56:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T07:56:58.255 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 394009 Links: 1 2023-10-07T07:56:58.255 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T07:56:58.255 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T07:56:58.255 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-10-07 07:56:52.525059544 +0000 2023-10-07T07:56:58.255 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-10-07 07:53:47.317781360 +0000 2023-10-07T07:56:58.255 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-10-07 07:53:47.317781360 +0000 2023-10-07T07:56:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2023-10-07T07:56:58.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-07T07:56:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-10-07T07:56:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-10-07T07:56:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000118487 s, 4.3 MB/s 2023-10-07T07:56:58.330 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-07T07:56:58.394 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2023-10-07T07:56:58.458 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-07T07:56:58.458 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T07:56:58.458 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 397921 Links: 1 2023-10-07T07:56:58.458 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T07:56:58.458 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T07:56:58.458 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-10-07 07:56:52.525059544 +0000 2023-10-07T07:56:58.458 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-10-07 07:53:47.880773014 +0000 2023-10-07T07:56:58.459 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-10-07 07:53:47.880773014 +0000 2023-10-07T07:56:58.459 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2023-10-07T07:56:58.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-07T07:56:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-10-07T07:56:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-10-07T07:56:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000131708 s, 3.9 MB/s 2023-10-07T07:56:58.533 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-07T07:56:58.598 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T07:56:58.598 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:56:58.599 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2023-10-07T07:56:58.621 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-07T07:56:58.621 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2023-10-07T07:56:58.683 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-07T07:56:58.684 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T07:56:58.684 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 395531 Links: 1 2023-10-07T07:56:58.684 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T07:56:58.684 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T07:56:58.684 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-10-07 07:56:44.390886329 +0000 2023-10-07T07:56:58.684 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-10-07 07:53:01.707756070 +0000 2023-10-07T07:56:58.684 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-10-07 07:53:01.707756070 +0000 2023-10-07T07:56:58.684 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-10-07T07:56:58.685 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-07T07:56:58.753 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-10-07T07:56:58.754 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-10-07T07:56:58.754 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000243261 s, 2.1 MB/s 2023-10-07T07:56:58.755 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-07T07:56:58.817 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2023-10-07T07:56:58.882 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-07T07:56:58.882 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T07:56:58.882 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 397515 Links: 1 2023-10-07T07:56:58.883 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T07:56:58.883 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T07:56:58.883 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-10-07 07:56:44.390886329 +0000 2023-10-07T07:56:58.883 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-10-07 07:53:02.254743224 +0000 2023-10-07T07:56:58.883 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-10-07 07:53:02.254743224 +0000 2023-10-07T07:56:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-10-07T07:56:58.883 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-07T07:56:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-10-07T07:56:58.951 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-10-07T07:56:58.951 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000228354 s, 2.2 MB/s 2023-10-07T07:56:58.952 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-07T07:56:59.012 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2023-10-07T07:56:59.071 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-07T07:56:59.071 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T07:56:59.071 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 398565 Links: 1 2023-10-07T07:56:59.072 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T07:56:59.072 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T07:56:59.072 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-10-07 07:56:44.390886329 +0000 2023-10-07T07:56:59.072 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-10-07 07:53:02.805730282 +0000 2023-10-07T07:56:59.072 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-10-07 07:53:02.805730282 +0000 2023-10-07T07:56:59.072 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-10-07T07:56:59.072 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-07T07:56:59.139 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-10-07T07:56:59.139 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-10-07T07:56:59.139 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00020118 s, 2.5 MB/s 2023-10-07T07:56:59.140 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-07T07:56:59.202 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2023-10-07T07:56:59.261 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-07T07:56:59.261 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T07:56:59.261 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 397619 Links: 1 2023-10-07T07:56:59.261 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T07:56:59.262 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T07:56:59.262 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-10-07 07:56:44.390886329 +0000 2023-10-07T07:56:59.262 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-10-07 07:53:03.316718279 +0000 2023-10-07T07:56:59.262 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-10-07 07:53:03.316718279 +0000 2023-10-07T07:56:59.262 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-10-07T07:56:59.262 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-07T07:56:59.328 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-10-07T07:56:59.328 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-10-07T07:56:59.328 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00028374 s, 1.8 MB/s 2023-10-07T07:56:59.329 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-07T07:56:59.391 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T07:56:59.392 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-07T07:56:59.392 INFO:tasks.ceph:Generating config... 2023-10-07T07:56:59.393 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-07T07:56:59.393 INFO:tasks.ceph:[client] debug client = 20 2023-10-07T07:56:59.393 INFO:tasks.ceph:[client] debug ms = 1 2023-10-07T07:56:59.393 INFO:tasks.ceph:[client] pid file = 2023-10-07T07:56:59.393 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-07T07:56:59.393 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-07T07:56:59.394 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-07T07:56:59.395 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-07T07:56:59.395 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-07T07:56:59.395 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-07T07:56:59.395 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-07T07:56:59.395 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-07T07:56:59.395 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-07T07:56:59.395 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-07T07:56:59.395 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-07T07:56:59.395 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-07T07:56:59.395 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-07T07:56:59.395 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-07T07:56:59.395 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-07T07:56:59.395 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-07T07:56:59.396 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-07T07:56:59.396 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-07T07:56:59.396 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-07T07:56:59.396 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-07T07:56:59.396 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-07T07:56:59.396 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-07T07:56:59.396 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-07T07:56:59.396 INFO:tasks.ceph:Setting up mon.a... 2023-10-07T07:56:59.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-07T07:56:59.455 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2023-10-07T07:56:59.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-07T07:56:59.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-07T07:56:59.541 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.172'), ('mon.c', '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]')] 2023-10-07T07:56:59.542 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-07T07:56:59.543 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:56:59.543 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-07T07:56:59.604 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.172 --addv c '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:56:59.700 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:56:59.700 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid c71d63e6-6de9-4139-83e7-b0bb7f6375d7 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = octopus 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:fsid c71d63e6-6de9-4139-83e7-b0bb7f6375d7 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2023-10-07T07:56:59.700621+0000 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:created 2023-10-07T07:56:59.700621+0000 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 15 (octopus) 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] mon.c 2023-10-07T07:56:59.701 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-07T07:56:59.703 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-07T07:56:59.725 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c71d63e6-6de9-4139-83e7-b0bb7f6375d7... 2023-10-07T07:56:59.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-07T07:56:59.767 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-07T07:56:59.816 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2023-10-07T07:56:59.816 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2023-10-07T07:56:59.816 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-07T07:56:59.816 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2023-10-07T07:56:59.816 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.816 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-07T07:56:59.817 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2023-10-07T07:56:59.818 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2023-10-07T07:56:59.819 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2023-10-07T07:56:59.819 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-07T07:56:59.819 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-07T07:56:59.819 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2023-10-07T07:56:59.819 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.819 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-07T07:56:59.819 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.819 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2023-10-07T07:56:59.819 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.7,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = c71d63e6-6de9-4139-83e7-b0bb7f6375d7 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.820 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2023-10-07T07:56:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2023-10-07T07:56:59.822 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: bdev async discard = True 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: bdev enable discard = True 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore allocator = bitmap 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 1/20 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 1/20 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 4/10 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2023-10-07T07:56:59.823 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2023-10-07T07:56:59.824 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2023-10-07T07:56:59.825 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2023-10-07T07:56:59.826 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-10-07T07:56:59.827 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2023-10-07T07:56:59.828 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-10-07T07:56:59.829 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2023-10-07T07:56:59.829 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = True 2023-10-07T07:56:59.829 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2023-10-07T07:56:59.829 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = True 2023-10-07T07:56:59.829 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2023-10-07T07:56:59.829 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2023-10-07T07:56:59.829 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2023-10-07T07:56:59.829 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2023-10-07T07:56:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2023-10-07T07:56:59.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-07T07:56:59.832 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = c71d63e6-6de9-4139-83e7-b0bb7f6375d7 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2023-10-07T07:56:59.833 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2023-10-07T07:56:59.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2023-10-07T07:56:59.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2023-10-07T07:56:59.836 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2023-10-07T07:56:59.837 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2023-10-07T07:56:59.838 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.839 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2023-10-07T07:56:59.840 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2023-10-07T07:56:59.841 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2023-10-07T07:56:59.842 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-10-07T07:56:59.842 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2023-10-07T07:56:59.842 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2023-10-07T07:56:59.842 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2023-10-07T07:56:59.842 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2023-10-07T07:56:59.842 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2023-10-07T07:56:59.842 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2023-10-07T07:56:59.842 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2023-10-07T07:56:59.843 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-07T07:56:59.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-07T07:56:59.950 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-07T07:56:59.951 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:56:59.951 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-07T07:56:59.972 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:56:59.972 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-07T07:57:00.035 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:57:00.035 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:00.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-07T07:57:00.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-07T07:57:00.123 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:00.123 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:00.186 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:57:00.186 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:00.186 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-07T07:57:00.186 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-07T07:57:00.228 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:00.228 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:00.289 INFO:tasks.ceph:Setting up mon nodes... 2023-10-07T07:57:00.289 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-07T07:57:00.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-07T07:57:00.352 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-07T07:57:00.355 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-07T07:57:00.413 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-07T07:57:00.416 INFO:tasks.ceph:Setting up mds nodes... 2023-10-07T07:57:00.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-07T07:57:00.480 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-07T07:57:00.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-07T07:57:00.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-07T07:57:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-07T07:57:00.620 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-07T07:57:00.652 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-07T07:57:00.705 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-07T07:57:00.707 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-07T07:57:00.735 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-07T07:57:00.825 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-07T07:57:00.827 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-07T07:57:00.854 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-07T07:57:00.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-07T07:57:00.905 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-07T07:57:00.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-07T07:57:00.968 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-07T07:57:00.982 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-07T07:57:01.024 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-07T07:57:01.036 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-07T07:57:01.116 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-07T07:57:01.128 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-07T07:57:01.128 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-07T07:57:01.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-07T07:57:01.163 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T07:57:01.163 INFO:tasks.ceph:role: osd.0 2023-10-07T07:57:01.163 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:57:01.163 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-07T07:57:01.238 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T07:57:01.238 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T07:57:01.238 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T07:57:01.238 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-07T07:57:01.238 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T07:57:01.238 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-10-07T07:57:01.238 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T07:57:01.239 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T07:57:01.239 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T07:57:01.239 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T07:57:01.261 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2023-10-07T07:57:01.263 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-10-07T07:57:01.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-07T07:57:01.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-07T07:57:01.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-07T07:57:01.467 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T07:57:01.467 INFO:tasks.ceph:role: osd.1 2023-10-07T07:57:01.467 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:57:01.467 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-07T07:57:01.542 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T07:57:01.542 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T07:57:01.542 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T07:57:01.542 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-07T07:57:01.542 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T07:57:01.543 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-10-07T07:57:01.543 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T07:57:01.543 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T07:57:01.543 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T07:57:01.543 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T07:57:01.564 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2023-10-07T07:57:01.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-10-07T07:57:01.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-07T07:57:01.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-07T07:57:01.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-07T07:57:01.769 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T07:57:01.769 INFO:tasks.ceph:role: osd.2 2023-10-07T07:57:01.769 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:57:01.769 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-07T07:57:01.843 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T07:57:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T07:57:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T07:57:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-07T07:57:01.844 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T07:57:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-10-07T07:57:01.844 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T07:57:01.844 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T07:57:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T07:57:01.844 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T07:57:01.866 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2023-10-07T07:57:01.868 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-10-07T07:57:01.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-07T07:57:01.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-07T07:57:01.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-07T07:57:02.067 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T07:57:02.068 INFO:tasks.ceph:role: osd.3 2023-10-07T07:57:02.068 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:57:02.068 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-07T07:57:02.143 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T07:57:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T07:57:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T07:57:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-07T07:57:02.143 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T07:57:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-10-07T07:57:02.143 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T07:57:02.144 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T07:57:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T07:57:02.144 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T07:57:02.167 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2023-10-07T07:57:02.168 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-10-07T07:57:02.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-07T07:57:02.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-07T07:57:02.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:02.432+0000 7feff90ce540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-07T07:57:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:02.432+0000 7feff90ce540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-07T07:57:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:02.433+0000 7feff90ce540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-07T07:57:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:02.433+0000 7feff90ce540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-07T07:57:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:02.433+0000 7feff90ce540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-07T07:57:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:02.433+0000 7feff90ce540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-07T07:57:02.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:02.957+0000 7feff90ce540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:03.721+0000 7feff90ce540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:04.483+0000 7feff90ce540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:05.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-07T07:57:05.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:05.409+0000 7f95aba21540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-07T07:57:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:05.410+0000 7f95aba21540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-07T07:57:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:05.410+0000 7f95aba21540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-07T07:57:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:05.410+0000 7f95aba21540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-07T07:57:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:05.410+0000 7f95aba21540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-07T07:57:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:05.410+0000 7f95aba21540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-07T07:57:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:05.938+0000 7f95aba21540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:06.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:06.701+0000 7f95aba21540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:07.463+0000 7f95aba21540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:08.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-07T07:57:08.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:08.394+0000 7f9afcf47540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-07T07:57:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:08.394+0000 7f9afcf47540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-07T07:57:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:08.394+0000 7f9afcf47540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-07T07:57:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:08.394+0000 7f9afcf47540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-07T07:57:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:08.394+0000 7f9afcf47540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-07T07:57:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:08.395+0000 7f9afcf47540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-07T07:57:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:08.921+0000 7f9afcf47540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:09.688+0000 7f9afcf47540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:10.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:10.452+0000 7f9afcf47540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:11.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-07T07:57:11.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:11.382+0000 7fa9d2ccb540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-07T07:57:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:11.382+0000 7fa9d2ccb540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-07T07:57:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:11.382+0000 7fa9d2ccb540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-07T07:57:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:11.383+0000 7fa9d2ccb540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-07T07:57:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:11.383+0000 7fa9d2ccb540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-07T07:57:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:11.383+0000 7fa9d2ccb540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-07T07:57:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:11.909+0000 7fa9d2ccb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:12.674+0000 7fa9d2ccb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:13.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:13.435+0000 7fa9d2ccb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:14.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-07T07:57:14.266 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-07T07:57:14.300 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T07:57:14.300 INFO:tasks.ceph:role: osd.4 2023-10-07T07:57:14.301 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:57:14.301 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-07T07:57:14.380 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T07:57:14.380 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T07:57:14.380 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T07:57:14.381 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-07T07:57:14.381 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T07:57:14.381 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-10-07T07:57:14.382 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T07:57:14.382 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T07:57:14.382 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T07:57:14.382 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T07:57:14.807 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-10-07T07:57:14.809 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-10-07T07:57:14.810 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-07T07:57:15.000 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-07T07:57:15.037 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-07T07:57:15.108 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T07:57:15.108 INFO:tasks.ceph:role: osd.5 2023-10-07T07:57:15.108 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:57:15.108 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-07T07:57:15.185 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T07:57:15.185 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T07:57:15.185 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T07:57:15.185 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-07T07:57:15.185 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T07:57:15.185 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-10-07T07:57:15.185 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T07:57:15.186 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T07:57:15.186 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T07:57:15.186 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T07:57:15.599 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-10-07T07:57:15.601 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-10-07T07:57:15.601 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-07T07:57:15.712 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-07T07:57:15.745 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-07T07:57:15.815 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T07:57:15.815 INFO:tasks.ceph:role: osd.6 2023-10-07T07:57:15.815 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:57:15.815 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-07T07:57:15.891 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T07:57:15.891 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T07:57:15.891 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T07:57:15.891 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-07T07:57:15.891 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T07:57:15.891 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-10-07T07:57:15.891 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T07:57:15.891 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T07:57:15.892 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T07:57:15.892 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T07:57:16.338 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-10-07T07:57:16.340 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-10-07T07:57:16.340 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-07T07:57:16.520 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-07T07:57:16.555 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-07T07:57:16.624 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T07:57:16.624 INFO:tasks.ceph:role: osd.7 2023-10-07T07:57:16.625 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:57:16.625 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-07T07:57:16.701 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T07:57:16.701 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T07:57:16.701 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T07:57:16.702 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-07T07:57:16.702 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T07:57:16.702 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-10-07T07:57:16.702 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T07:57:16.702 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T07:57:16.702 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T07:57:16.702 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T07:57:17.137 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-10-07T07:57:17.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-10-07T07:57:17.139 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-07T07:57:17.320 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-07T07:57:17.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:17.450 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:17.449+0000 7f5fac1f1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-07T07:57:17.451 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:17.451+0000 7f5fac1f1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-07T07:57:17.451 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:17.451+0000 7f5fac1f1540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-07T07:57:17.451 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:17.451+0000 7f5fac1f1540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-07T07:57:17.451 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:17.451+0000 7f5fac1f1540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-07T07:57:17.452 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:17.451+0000 7f5fac1f1540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-07T07:57:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:17.980+0000 7f5fac1f1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:18.744 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:18.743+0000 7f5fac1f1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:19.506 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:19.506+0000 7f5fac1f1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:20.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-07T07:57:20.338 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:20.429 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:20.428+0000 7fca48bf3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-07T07:57:20.429 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:20.428+0000 7fca48bf3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-07T07:57:20.429 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:20.428+0000 7fca48bf3540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-07T07:57:20.429 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:20.428+0000 7fca48bf3540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-07T07:57:20.429 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:20.428+0000 7fca48bf3540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-07T07:57:20.429 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:20.429+0000 7fca48bf3540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-07T07:57:20.955 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:20.954+0000 7fca48bf3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:21.717+0000 7fca48bf3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:22.480 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:22.478+0000 7fca48bf3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:23.274 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-07T07:57:23.307 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:23.398 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:23.396+0000 7fe93f22d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-07T07:57:23.398 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:23.396+0000 7fe93f22d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-07T07:57:23.398 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:23.396+0000 7fe93f22d540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-07T07:57:23.398 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:23.396+0000 7fe93f22d540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-07T07:57:23.398 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:23.396+0000 7fe93f22d540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-07T07:57:23.398 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:23.397+0000 7fe93f22d540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-07T07:57:23.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:23.928+0000 7fe93f22d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:24.692 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:24.691+0000 7fe93f22d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:25.460 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:25.458+0000 7fe93f22d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:26.254 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-07T07:57:26.287 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:26.375 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:26.373+0000 7f1eaddae540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-07T07:57:26.375 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:26.374+0000 7f1eaddae540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-07T07:57:26.375 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:26.374+0000 7f1eaddae540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-07T07:57:26.375 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:26.374+0000 7f1eaddae540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-07T07:57:26.375 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:26.374+0000 7f1eaddae540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-07T07:57:26.376 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:26.374+0000 7f1eaddae540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-07T07:57:26.900 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:26.899+0000 7f1eaddae540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:27.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:57:27.595 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:57:27.672 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:27.671+0000 7f1eaddae540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:28.435 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:28.434+0000 7f1eaddae540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T07:57:29.229 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-07T07:57:29.303 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-07T07:57:29.303 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:29.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-07T07:57:29.338 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:29.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-07T07:57:29.414 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:29.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-07T07:57:29.489 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:29.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-07T07:57:29.564 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:29.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-07T07:57:29.639 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:29.640 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-07T07:57:29.715 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:29.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-07T07:57:29.790 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:29.790 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-07T07:57:29.823 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:29.823 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-07T07:57:29.895 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:29.895 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-07T07:57:29.965 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:29.965 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-07T07:57:30.034 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:30.034 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-07T07:57:30.102 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:30.102 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-07T07:57:30.169 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:30.170 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-07T07:57:30.238 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:30.238 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-07T07:57:30.261 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:30.261 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-07T07:57:30.324 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:30.325 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-07T07:57:30.343 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:30.343 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-07T07:57:30.401 INFO:tasks.ceph:Adding keys to all mons... 2023-10-07T07:57:30.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-07T07:57:30.404 DEBUG:teuthology.orchestra.run.smithi172:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-07T07:57:30.444 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2023-10-07T07:57:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBMDyFlhkIGFRAAWzKzlNAmIb3PXAlOaZMtYA== 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBMDyFlCy6iHBAA0pyZcla7yG20KJxMwhhFVg== 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBMDyFluHPbJBAAdTCLab+QEn7jhGF253MA4A== 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBODyFlkgzdGRAAUXBLhY69mpInyEg/EZ1bbg== 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBRDyFl95iAGBAA8mk47fzfwi2BDWqxoJIAGA== 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBUDyFlpgWXFxAABhoYDIRsLPfY19Z76zlRfg== 2023-10-07T07:57:30.445 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBXDyFl9VThFhAAKbJJRsjjubjWK5xpt7j6Vg== 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBMDyFlgeSpGBAAlq+bBWoPBk7KPt5+xL1z4w== 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBMDyFlUxYOKhAAuXUwfiu6GTlHPau/HLhgJg== 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBMDyFlNaU3MRAAqpmLriqCxUsXuAyL5Cmbxw== 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBdDyFlcKXTGhAAPW0NNpqt2aoxJZUSdBtJEw== 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2023-10-07T07:57:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBgDyFlYdmOGRAAO4UiW/o3zkFHKLo0A/Fl0g== 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBjDyFlxgq2FxAAar/DNRzL+QMVm0sZJ6dY1A== 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBmDyFlZyNcFhAAfEAyrTMB6AkewTly3i3vVQ== 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBMDyFlom77NRAA6mby9LFc/5XVOSXctSuffA== 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:[client.2] 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBMDyFlrCK0ORAAWpaeoXzMPo3ko8Yrat1bEg== 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBNDyFlGLtzARAAh3o32z7sOz1RBqXINgae6g== 2023-10-07T07:57:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:[client.3] 2023-10-07T07:57:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBNDyFl/YjxBhAAivYmiSK5+fdwyFmyMeWI+A== 2023-10-07T07:57:30.467 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.y] 2023-10-07T07:57:30.467 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBMDyFlhkIGFRAAWzKzlNAmIb3PXAlOaZMtYA== 2023-10-07T07:57:30.467 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2023-10-07T07:57:30.467 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBMDyFlCy6iHBAA0pyZcla7yG20KJxMwhhFVg== 2023-10-07T07:57:30.467 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2023-10-07T07:57:30.467 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBMDyFluHPbJBAAdTCLab+QEn7jhGF253MA4A== 2023-10-07T07:57:30.467 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.0] 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBODyFlkgzdGRAAUXBLhY69mpInyEg/EZ1bbg== 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.1] 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBRDyFl95iAGBAA8mk47fzfwi2BDWqxoJIAGA== 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.2] 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBUDyFlpgWXFxAABhoYDIRsLPfY19Z76zlRfg== 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.3] 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBXDyFl9VThFhAAKbJJRsjjubjWK5xpt7j6Vg== 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.x] 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBMDyFlgeSpGBAAlq+bBWoPBk7KPt5+xL1z4w== 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBMDyFlUxYOKhAAuXUwfiu6GTlHPau/HLhgJg== 2023-10-07T07:57:30.468 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBMDyFlNaU3MRAAqpmLriqCxUsXuAyL5Cmbxw== 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.4] 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBdDyFlcKXTGhAAPW0NNpqt2aoxJZUSdBtJEw== 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.5] 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBgDyFlYdmOGRAAO4UiW/o3zkFHKLo0A/Fl0g== 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.6] 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBjDyFlxgq2FxAAar/DNRzL+QMVm0sZJ6dY1A== 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.7] 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBmDyFlZyNcFhAAfEAyrTMB6AkewTly3i3vVQ== 2023-10-07T07:57:30.469 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2023-10-07T07:57:30.470 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBMDyFlom77NRAA6mby9LFc/5XVOSXctSuffA== 2023-10-07T07:57:30.470 INFO:teuthology.orchestra.run.smithi172.stdout:[client.2] 2023-10-07T07:57:30.470 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBMDyFlrCK0ORAAWpaeoXzMPo3ko8Yrat1bEg== 2023-10-07T07:57:30.470 INFO:teuthology.orchestra.run.smithi172.stdout:[client.1] 2023-10-07T07:57:30.470 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBNDyFlGLtzARAAh3o32z7sOz1RBqXINgae6g== 2023-10-07T07:57:30.470 INFO:teuthology.orchestra.run.smithi172.stdout:[client.3] 2023-10-07T07:57:30.470 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBNDyFl/YjxBhAAivYmiSK5+fdwyFmyMeWI+A== 2023-10-07T07:57:30.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-07T07:57:30.492 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-07T07:57:30.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T07:57:30.566 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T07:57:30.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T07:57:30.628 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T07:57:30.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:30.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:30.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:30.745 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:30.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:30.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:30.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:30.857 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:30.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-07T07:57:30.913 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-07T07:57:30.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T07:57:30.969 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T07:57:31.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T07:57:31.028 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T07:57:31.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:31.086 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:31.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:31.142 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:31.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:31.198 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:31.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:31.253 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T07:57:31.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T07:57:31.309 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T07:57:31.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T07:57:31.369 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T07:57:31.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T07:57:31.428 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T07:57:31.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T07:57:31.487 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T07:57:31.543 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-07T07:57:31.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-07T07:57:31.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-07T07:57:31.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-07T07:57:31.712 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-07T07:57:31.740 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-07T07:57:31.835 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-07T07:57:31.863 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-07T07:57:31.930 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-07T07:57:32.024 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-07T07:57:32.052 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:32.054 DEBUG:teuthology.orchestra.run.smithi172:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-07T07:57:32.110 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-07T07:57:32.110 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-07T07:57:32.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-07T07:57:32.117 INFO:tasks.ceph.mon.a:Started 2023-10-07T07:57:32.118 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-07T07:57:32.118 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-07T07:57:32.152 INFO:tasks.ceph.mon.b:Started 2023-10-07T07:57:32.153 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-07T07:57:32.153 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-07T07:57:32.154 INFO:tasks.ceph.mon.c:Started 2023-10-07T07:57:32.154 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-07T07:57:32.155 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-07T07:57:32.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-07T07:57:32.157 INFO:tasks.ceph.mgr.y:Started 2023-10-07T07:57:32.157 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-07T07:57:32.157 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-07T07:57:32.159 INFO:tasks.ceph.mgr.x:Started 2023-10-07T07:57:32.160 INFO:tasks.ceph:Setting crush tunables to default 2023-10-07T07:57:32.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2023-10-07T07:57:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.296+0000 7fa8d7957700 1 Processor -- start 2023-10-07T07:57:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.296+0000 7fa8d7957700 1 -- start start 2023-10-07T07:57:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d7957700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 0x7fa8d00cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d7957700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa8d015dbf0 0x7fa8d00afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d7957700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d00ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d7957700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d00cf1e0 con 0x7fa8d015dbf0 2023-10-07T07:57:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d7957700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d00ce5d0 con 0x7fa8d00af440 2023-10-07T07:57:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d7957700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00ce8b0 con 0x7fa8d00af810 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d4ef2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa8d015dbf0 0x7fa8d00afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d4ef2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa8d015dbf0 0x7fa8d00afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40088/0 (socket says 172.21.15.7:40088) 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d4ef2700 1 -- 172.21.15.7:0/1117958192 learned_addr learned my addr 172.21.15.7:0/1117958192 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d4ef2700 1 -- 172.21.15.7:0/1117958192 >> v2:172.21.15.7:3300/0 conn(0x7fa8d015dbf0 msgr2=0x7fa8d00afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d4ef2700 1 -- 172.21.15.7:0/1117958192 >> v2:172.21.15.7:3300/0 conn(0x7fa8d015dbf0 msgr2=0x7fa8d00afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/1117958192 >> v2:172.21.15.7:3300/0 conn(0x7fa8d015dbf0 0x7fa8d00afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/1117958192 >> v2:172.21.15.7:3300/0 conn(0x7fa8d015dbf0 0x7fa8d00afd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d5ef4700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d00ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d56f3700 1 --2- 172.21.15.7:0/1117958192 >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 0x7fa8d00cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d5ef4700 1 -- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 msgr2=0x7fa8d00ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-07T07:57:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d5ef4700 1 -- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 msgr2=0x7fa8d00ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T07:57:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d5ef4700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d00ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T07:57:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d56f3700 1 -- 172.21.15.7:0/1117958192 >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 msgr2=0x7fa8d00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-07T07:57:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d56f3700 1 -- 172.21.15.7:0/1117958192 >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 msgr2=0x7fa8d00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T07:57:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d5ef4700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d00ccc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T07:57:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d56f3700 1 --2- 172.21.15.7:0/1117958192 >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 0x7fa8d00cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T07:57:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.297+0000 7fa8d56f3700 1 --2- 172.21.15.7:0/1117958192 >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 0x7fa8d00cb2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T07:57:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/1117958192 >> v2:172.21.15.7:3300/0 conn(0x7fa8d015dbf0 0x7fa8d00afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d5ef4700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d00ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d56f3700 1 --2- 172.21.15.7:0/1117958192 >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 0x7fa8d00cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d4ef2700 1 -- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 msgr2=0x7fa8d00ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d00ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d4ef2700 1 -- 172.21.15.7:0/1117958192 >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 msgr2=0x7fa8d00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/1117958192 >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 0x7fa8d00cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d4ef2700 1 -- 172.21.15.7:0/1117958192 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00ceb90 con 0x7fa8d015dbf0 2023-10-07T07:57:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d5ef4700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d00ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.498+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d015dbf0 0x7fa8d00afd00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00678c0 tx=0x7fa8c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ff9f985e91a373 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.499+0000 7fa8cffff700 1 -- 172.21.15.7:0/1117958192 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400e490 con 0x7fa8d015dbf0 2023-10-07T07:57:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.499+0000 7fa8cffff700 1 -- 172.21.15.7:0/1117958192 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c400e5f0 con 0x7fa8d015dbf0 2023-10-07T07:57:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.499+0000 7fa8cffff700 1 -- 172.21.15.7:0/1117958192 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400e9f0 con 0x7fa8d015dbf0 2023-10-07T07:57:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.500+0000 7fa8d7957700 1 -- 172.21.15.7:0/1117958192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d015dbf0 msgr2=0x7fa8d00afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.500+0000 7fa8d7957700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d015dbf0 0x7fa8d00afd00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00678c0 tx=0x7fa8c4008040 comp rx=0 tx=0).stop 2023-10-07T07:57:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.500+0000 7fa8d7957700 1 -- 172.21.15.7:0/1117958192 shutdown_connections 2023-10-07T07:57:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.500+0000 7fa8d7957700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d00ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.500+0000 7fa8d7957700 1 --2- 172.21.15.7:0/1117958192 >> v2:172.21.15.172:3300/0 conn(0x7fa8d00af440 0x7fa8d00cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.500+0000 7fa8d7957700 1 --2- 172.21.15.7:0/1117958192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d015dbf0 0x7fa8d00afd00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.500+0000 7fa8d7957700 1 -- 172.21.15.7:0/1117958192 >> 172.21.15.7:0/1117958192 conn(0x7fa8d0069220 msgr2=0x7fa8d0069620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.500+0000 7fa8d7957700 1 -- 172.21.15.7:0/1117958192 shutdown_connections 2023-10-07T07:57:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.500+0000 7fa8d7957700 1 -- 172.21.15.7:0/1117958192 wait complete. 2023-10-07T07:57:32.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.502+0000 7fa8d7957700 1 Processor -- start 2023-10-07T07:57:32.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.502+0000 7fa8d7957700 1 -- start start 2023-10-07T07:57:32.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.503+0000 7fa8d7957700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d00af440 0x7fa8d014fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:32.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.503+0000 7fa8d7957700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d0153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:32.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.503+0000 7fa8d7957700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d015dbf0 0x7fa8d0169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d7957700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d01501f0 con 0x7fa8d00af440 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d7957700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0159b60 con 0x7fa8d015dbf0 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d7957700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0159e40 con 0x7fa8d00af810 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d56f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d00af440 0x7fa8d014fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d56f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d00af440 0x7fa8d014fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40134/0 (socket says 172.21.15.7:40134) 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d56f3700 1 -- 172.21.15.7:0/667187957 learned_addr learned my addr 172.21.15.7:0/667187957 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d0153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d5ef4700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d015dbf0 0x7fa8d0169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d4ef2700 1 -- 172.21.15.7:0/667187957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d015dbf0 msgr2=0x7fa8d0169ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d015dbf0 0x7fa8d0169ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d4ef2700 1 -- 172.21.15.7:0/667187957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d00af440 msgr2=0x7fa8d014fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d00af440 0x7fa8d014fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d4ef2700 1 -- 172.21.15.7:0/667187957 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8009a80 con 0x7fa8d00af810 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d56f3700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d00af440 0x7fa8d014fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8d4ef2700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d0153ca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0069fe0 tx=0x7fa8c401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.504+0000 7fa8cd7fa700 1 -- 172.21.15.7:0/667187957 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4013920 con 0x7fa8d00af810 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.505+0000 7fa8cd7fa700 1 -- 172.21.15.7:0/667187957 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c4013ae0 con 0x7fa8d00af810 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.505+0000 7fa8cd7fa700 1 -- 172.21.15.7:0/667187957 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4024410 con 0x7fa8d00af810 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.505+0000 7fa8d7957700 1 -- 172.21.15.7:0/667187957 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c4009570 con 0x7fa8d00af810 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.505+0000 7fa8d7957700 1 -- 172.21.15.7:0/667187957 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d015a400 con 0x7fa8d00af810 2023-10-07T07:57:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.505+0000 7fa8cd7fa700 1 -- 172.21.15.7:0/667187957 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa8c4024840 con 0x7fa8d00af810 2023-10-07T07:57:32.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.505+0000 7fa8cd7fa700 1 -- 172.21.15.7:0/667187957 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa8c402d400 con 0x7fa8d00af810 2023-10-07T07:57:32.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.506+0000 7fa8d7957700 1 -- 172.21.15.7:0/667187957 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d015a5f0 con 0x7fa8d00af810 2023-10-07T07:57:32.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.508+0000 7fa8cd7fa700 1 -- 172.21.15.7:0/667187957 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa8c400c070 con 0x7fa8d00af810 2023-10-07T07:57:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.590+0000 7fa8d7957700 1 -- 172.21.15.7:0/667187957 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa8d004c3c0 con 0x7fa8d00af810 2023-10-07T07:57:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.593+0000 7fa8cd7fa700 1 -- 172.21.15.7:0/667187957 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa8c400f040 con 0x7fa8d00af810 2023-10-07T07:57:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.594+0000 7fa8d7957700 1 -- 172.21.15.7:0/667187957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 msgr2=0x7fa8d0153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.594+0000 7fa8d7957700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d0153ca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0069fe0 tx=0x7fa8c401a000 comp rx=0 tx=0).stop 2023-10-07T07:57:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.594+0000 7fa8d7957700 1 -- 172.21.15.7:0/667187957 shutdown_connections 2023-10-07T07:57:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.594+0000 7fa8d7957700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d015dbf0 0x7fa8d0169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.594+0000 7fa8d7957700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8d00af440 0x7fa8d014fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.594+0000 7fa8d7957700 1 --2- 172.21.15.7:0/667187957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8d00af810 0x7fa8d0153ca0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.594+0000 7fa8d7957700 1 -- 172.21.15.7:0/667187957 >> 172.21.15.7:0/667187957 conn(0x7fa8d0069220 msgr2=0x7fa8d00afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.594+0000 7fa8d7957700 1 -- 172.21.15.7:0/667187957 shutdown_connections 2023-10-07T07:57:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:32.594+0000 7fa8d7957700 1 -- 172.21.15.7:0/667187957 wait complete. 2023-10-07T07:57:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2023-10-07T07:57:32.609 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-07T07:57:32.609 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:32.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-07T07:57:32.610 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:32.608+0000 7f8a905001c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-07T07:57:32.635 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:32.634+0000 7f9216a671c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-07T07:57:32.639 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:32.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-07T07:57:32.713 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:32.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-07T07:57:32.775 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:32.774+0000 7f8a905001c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-07T07:57:32.811 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-10-07T07:57:32.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-07T07:57:32.831 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:32.830+0000 7f9216a671c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-07T07:57:32.841 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:32.841 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-07T07:57:32.870 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:32.870 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-07T07:57:32.937 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:32.938 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-07T07:57:33.005 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-10-07T07:57:33.005 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-07T07:57:33.073 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 26598042-733a-424c-a5e5-3b75a67a460e 0 2023-10-07T07:57:33.132 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:33.131+0000 7f8a905001c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-07T07:57:33.203 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:33.202+0000 7f9216a671c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-07T07:57:33.233 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.232+0000 7feac7237700 1 Processor -- start 2023-10-07T07:57:33.233 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.232+0000 7feac7237700 1 -- start start 2023-10-07T07:57:33.235 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac7237700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feac0157540 0x7feac0068000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac7237700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feac0068540 0x7feac014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac7237700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac00c6e00 con 0x7feac0157170 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac7237700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac0068960 con 0x7feac0157540 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac7237700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac006b8f0 con 0x7feac0068540 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac57d4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feac0068540 0x7feac014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac4fd3700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7feac0157170 0x7feac006aef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54332/0 (socket says 172.21.15.172:54332) 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac4fd3700 1 -- 172.21.15.172:0/3767412878 learned_addr learned my addr 172.21.15.172:0/3767412878 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feabffff700 1 --2- 172.21.15.172:0/3767412878 >> v2:172.21.15.7:3300/0 conn(0x7feac0157540 0x7feac0068000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac57d4700 1 -- 172.21.15.172:0/3767412878 >> v2:172.21.15.7:3300/0 conn(0x7feac0157540 msgr2=0x7feac0068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:33.236 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac57d4700 1 --2- 172.21.15.172:0/3767412878 >> v2:172.21.15.7:3300/0 conn(0x7feac0157540 0x7feac0068000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac57d4700 1 -- 172.21.15.172:0/3767412878 >> v1:172.21.15.172:6789/0 conn(0x7feac0157170 legacy=0x7feac006aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.233+0000 7feac57d4700 1 -- 172.21.15.172:0/3767412878 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac006bbd0 con 0x7feac0068540 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feac57d4700 1 --2- 172.21.15.172:0/3767412878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0068540 0x7feac014cb90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feac00c6d00 tx=0x7feab8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b453ba88f8c3ee5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feabf7fe700 1 -- 172.21.15.172:0/3767412878 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab800e070 con 0x7feac0068540 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feabf7fe700 1 -- 172.21.15.172:0/3767412878 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab800bc30 con 0x7feac0068540 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feabf7fe700 1 -- 172.21.15.172:0/3767412878 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8012430 con 0x7feac0068540 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feac7237700 1 -- 172.21.15.172:0/3767412878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0068540 msgr2=0x7feac014cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feac7237700 1 --2- 172.21.15.172:0/3767412878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0068540 0x7feac014cb90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feac00c6d00 tx=0x7feab8009780 comp rx=0 tx=0).stop 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feac7237700 1 -- 172.21.15.172:0/3767412878 shutdown_connections 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feac7237700 1 --2- 172.21.15.172:0/3767412878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0068540 0x7feac014cb90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feac7237700 1 --2- 172.21.15.172:0/3767412878 >> v2:172.21.15.7:3300/0 conn(0x7feac0157540 0x7feac0068000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.237 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feac7237700 1 -- 172.21.15.172:0/3767412878 >> 172.21.15.172:0/3767412878 conn(0x7feac00c9c30 msgr2=0x7feac006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:33.238 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feac7237700 1 -- 172.21.15.172:0/3767412878 shutdown_connections 2023-10-07T07:57:33.238 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.234+0000 7feac7237700 1 -- 172.21.15.172:0/3767412878 wait complete. 2023-10-07T07:57:33.240 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.235+0000 7feac7237700 1 Processor -- start 2023-10-07T07:57:33.240 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.236+0000 7feac7237700 1 -- start start 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.236+0000 7feac7237700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac0068540 0x7feac0144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.236+0000 7feac7237700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feac0157170 0x7feac014e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.236+0000 7feac7237700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0157540 0x7feac015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.236+0000 7feac7237700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00c1d50 con 0x7feac0068540 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.236+0000 7feac7237700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac014fbb0 con 0x7feac0157540 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.236+0000 7feac7237700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac014f310 con 0x7feac0157170 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac57d4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0157540 0x7feac015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feabffff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feac0157170 0x7feac014e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feabffff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feac0157170 0x7feac014e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:40130/0 (socket says 172.21.15.172:40130) 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac57d4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0157540 0x7feac015bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53668/0 (socket says 172.21.15.172:53668) 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feabffff700 1 -- 172.21.15.172:0/3164787583 learned_addr learned my addr 172.21.15.172:0/3164787583 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:33.241 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac4fd3700 1 --2- 172.21.15.172:0/3164787583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac0068540 0x7feac0144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac57d4700 1 -- 172.21.15.172:0/3164787583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feac0157170 msgr2=0x7feac014e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac57d4700 1 --2- 172.21.15.172:0/3164787583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feac0157170 0x7feac014e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac57d4700 1 -- 172.21.15.172:0/3164787583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac0068540 msgr2=0x7feac0144580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac57d4700 1 --2- 172.21.15.172:0/3164787583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac0068540 0x7feac0144580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac57d4700 1 -- 172.21.15.172:0/3164787583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8009a20 con 0x7feac0157540 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac57d4700 1 --2- 172.21.15.172:0/3164787583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0157540 0x7feac015bd20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feab8016000 tx=0x7feab8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feabcff9700 1 -- 172.21.15.172:0/3164787583 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8027470 con 0x7feac0157540 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feabcff9700 1 -- 172.21.15.172:0/3164787583 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab8027630 con 0x7feac0157540 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feabcff9700 1 -- 172.21.15.172:0/3164787583 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab80279f0 con 0x7feac0157540 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac7237700 1 -- 172.21.15.172:0/3164787583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac014f5f0 con 0x7feac0157540 2023-10-07T07:57:33.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feac7237700 1 -- 172.21.15.172:0/3164787583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac014eee0 con 0x7feac0157540 2023-10-07T07:57:33.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feabcff9700 1 -- 172.21.15.172:0/3164787583 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7feab8024070 con 0x7feac0157540 2023-10-07T07:57:33.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.237+0000 7feabcff9700 1 -- 172.21.15.172:0/3164787583 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7feab801e6f0 con 0x7feac0157540 2023-10-07T07:57:33.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.238+0000 7feac7237700 1 -- 172.21.15.172:0/3164787583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac014f0d0 con 0x7feac0157540 2023-10-07T07:57:33.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.242+0000 7feabcff9700 1 -- 172.21.15.172:0/3164787583 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7feab80476d0 con 0x7feac0157540 2023-10-07T07:57:33.327 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.325+0000 7feac7237700 1 -- 172.21.15.172:0/3164787583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "26598042-733a-424c-a5e5-3b75a67a460e", "id": 0} v 0) v1 -- 0x7feac00c1b20 con 0x7feac0157540 2023-10-07T07:57:33.331 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.329+0000 7feabcff9700 1 -- 172.21.15.172:0/3164787583 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "26598042-733a-424c-a5e5-3b75a67a460e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7feac00c1b20 con 0x7feac0157540 2023-10-07T07:57:33.331 INFO:teuthology.orchestra.run.smithi172.stdout:0 2023-10-07T07:57:33.332 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.330+0000 7feac7237700 1 -- 172.21.15.172:0/3164787583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0157540 msgr2=0x7feac015bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.332 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.330+0000 7feac7237700 1 --2- 172.21.15.172:0/3164787583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0157540 0x7feac015bd20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feab8016000 tx=0x7feab8019000 comp rx=0 tx=0).stop 2023-10-07T07:57:33.332 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.330+0000 7feac7237700 1 -- 172.21.15.172:0/3164787583 shutdown_connections 2023-10-07T07:57:33.332 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.330+0000 7feac7237700 1 --2- 172.21.15.172:0/3164787583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feac0157540 0x7feac015bd20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.332 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.330+0000 7feac7237700 1 --2- 172.21.15.172:0/3164787583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac0068540 0x7feac0144580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.332 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.330+0000 7feac7237700 1 --2- 172.21.15.172:0/3164787583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feac0157170 0x7feac014e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.332 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.330+0000 7feac7237700 1 -- 172.21.15.172:0/3164787583 >> 172.21.15.172:0/3164787583 conn(0x7feac00c9c30 msgr2=0x7feac0154410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:33.332 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.330+0000 7feac7237700 1 -- 172.21.15.172:0/3164787583 shutdown_connections 2023-10-07T07:57:33.332 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.330+0000 7feac7237700 1 -- 172.21.15.172:0/3164787583 wait complete. 2023-10-07T07:57:33.343 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 26aad281-717c-4bde-8dca-873b8714ac4e 1 2023-10-07T07:57:33.504 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.503+0000 7f4f25ae4700 1 Processor -- start 2023-10-07T07:57:33.504 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.503+0000 7f4f25ae4700 1 -- start start 2023-10-07T07:57:33.507 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.503+0000 7f4f25ae4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4f2015c920 0x7f4f200ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.507 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.503+0000 7f4f25ae4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f20068710 0x7f4f2006c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.507 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.503+0000 7f4f25ae4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2006c600 con 0x7f4f2015ccf0 2023-10-07T07:57:33.507 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.503+0000 7f4f25ae4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f200cb3c0 con 0x7f4f20068710 2023-10-07T07:57:33.507 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.503+0000 7f4f25ae4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f200cab00 con 0x7f4f2015c920 2023-10-07T07:57:33.507 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1f7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4f2015c920 0x7f4f200ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.507 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1effd700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4f2015ccf0 0x7f4f20068000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54342/0 (socket says 172.21.15.172:54342) 2023-10-07T07:57:33.507 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1effd700 1 -- 172.21.15.172:0/2985451086 learned_addr learned my addr 172.21.15.172:0/2985451086 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-10-07T07:57:33.507 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1ffff700 1 --2- 172.21.15.172:0/2985451086 >> v2:172.21.15.7:3300/0 conn(0x7f4f20068710 0x7f4f2006c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.508 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1f7fe700 1 -- 172.21.15.172:0/2985451086 >> v2:172.21.15.7:3300/0 conn(0x7f4f20068710 msgr2=0x7f4f2006c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:33.508 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1f7fe700 1 --2- 172.21.15.172:0/2985451086 >> v2:172.21.15.7:3300/0 conn(0x7f4f20068710 0x7f4f2006c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.508 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1f7fe700 1 -- 172.21.15.172:0/2985451086 >> v1:172.21.15.172:6789/0 conn(0x7f4f2015ccf0 legacy=0x7f4f20068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.508 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1f7fe700 1 -- 172.21.15.172:0/2985451086 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2006a7a0 con 0x7f4f2015c920 2023-10-07T07:57:33.508 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1e7fc700 1 -- 172.21.15.172:0/2985451086 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462164544 0 0) 0x7f4f14001110 con 0x7f4f2015ccf0 2023-10-07T07:57:33.508 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1ffff700 1 --2- 172.21.15.172:0/2985451086 >> v2:172.21.15.7:3300/0 conn(0x7f4f20068710 0x7f4f2006c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:57:33.508 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1f7fe700 1 --2- 172.21.15.172:0/2985451086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f2015c920 0x7f4f200ca520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4f200c3790 tx=0x7f4f10009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f45816f1b54e10e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:33.508 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1e7fc700 1 -- 172.21.15.172:0/2985451086 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10005c10 con 0x7f4f2015c920 2023-10-07T07:57:33.508 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1e7fc700 1 -- 172.21.15.172:0/2985451086 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f10005d70 con 0x7f4f2015c920 2023-10-07T07:57:33.509 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.504+0000 7f4f1e7fc700 1 -- 172.21.15.172:0/2985451086 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10006280 con 0x7f4f2015c920 2023-10-07T07:57:33.509 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.505+0000 7f4f25ae4700 1 -- 172.21.15.172:0/2985451086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f2015c920 msgr2=0x7f4f200ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.509 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.505+0000 7f4f25ae4700 1 --2- 172.21.15.172:0/2985451086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f2015c920 0x7f4f200ca520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4f200c3790 tx=0x7f4f10009120 comp rx=0 tx=0).stop 2023-10-07T07:57:33.509 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.505+0000 7f4f25ae4700 1 -- 172.21.15.172:0/2985451086 shutdown_connections 2023-10-07T07:57:33.509 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.505+0000 7f4f25ae4700 1 --2- 172.21.15.172:0/2985451086 >> v2:172.21.15.7:3300/0 conn(0x7f4f20068710 0x7f4f2006c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.509 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.505+0000 7f4f25ae4700 1 --2- 172.21.15.172:0/2985451086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f2015c920 0x7f4f200ca520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.509 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.505+0000 7f4f25ae4700 1 -- 172.21.15.172:0/2985451086 >> 172.21.15.172:0/2985451086 conn(0x7f4f200c9c30 msgr2=0x7f4f2006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:33.509 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.505+0000 7f4f25ae4700 1 -- 172.21.15.172:0/2985451086 shutdown_connections 2023-10-07T07:57:33.509 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.505+0000 7f4f25ae4700 1 -- 172.21.15.172:0/2985451086 wait complete. 2023-10-07T07:57:33.513 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.506+0000 7f4f25ae4700 1 Processor -- start 2023-10-07T07:57:33.513 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.506+0000 7f4f25ae4700 1 -- start start 2023-10-07T07:57:33.513 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f25ae4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20068710 0x7f4f2014f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.513 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f25ae4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2015c920 0x7f4f200b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f25ae4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2015ccf0 0x7f4f200c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f25ae4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f20158b60 con 0x7f4f2015c920 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f25ae4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f200c39d0 con 0x7f4f20068710 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f25ae4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f200b1910 con 0x7f4f2015ccf0 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1ffff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2015ccf0 0x7f4f200c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1f7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20068710 0x7f4f2014f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1ffff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2015ccf0 0x7f4f200c2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:40144/0 (socket says 172.21.15.172:40144) 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1ffff700 1 -- 172.21.15.172:0/3151917340 learned_addr learned my addr 172.21.15.172:0/3151917340 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1f7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20068710 0x7f4f2014f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53688/0 (socket says 172.21.15.172:53688) 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1effd700 1 --2- 172.21.15.172:0/3151917340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2015c920 0x7f4f200b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1f7fe700 1 -- 172.21.15.172:0/3151917340 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2015ccf0 msgr2=0x7f4f200c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.514 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1f7fe700 1 --2- 172.21.15.172:0/3151917340 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2015ccf0 0x7f4f200c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1f7fe700 1 -- 172.21.15.172:0/3151917340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2015c920 msgr2=0x7f4f200b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1f7fe700 1 --2- 172.21.15.172:0/3151917340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2015c920 0x7f4f200b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1f7fe700 1 -- 172.21.15.172:0/3151917340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f100093c0 con 0x7f4f20068710 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f1f7fe700 1 --2- 172.21.15.172:0/3151917340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20068710 0x7f4f2014f8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4f10015000 tx=0x7f4f10016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.507+0000 7f4f077fe700 1 -- 172.21.15.172:0/3151917340 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10006a80 con 0x7f4f20068710 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.508+0000 7f4f077fe700 1 -- 172.21.15.172:0/3151917340 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f10006210 con 0x7f4f20068710 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.508+0000 7f4f25ae4700 1 -- 172.21.15.172:0/3151917340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f200b1bf0 con 0x7f4f20068710 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.508+0000 7f4f077fe700 1 -- 172.21.15.172:0/3151917340 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000bde0 con 0x7f4f20068710 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.508+0000 7f4f25ae4700 1 -- 172.21.15.172:0/3151917340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f200b21b0 con 0x7f4f20068710 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.508+0000 7f4f077fe700 1 -- 172.21.15.172:0/3151917340 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4f1001b910 con 0x7f4f20068710 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.508+0000 7f4f077fe700 1 -- 172.21.15.172:0/3151917340 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f4f1002f480 con 0x7f4f20068710 2023-10-07T07:57:33.515 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.509+0000 7f4f25ae4700 1 -- 172.21.15.172:0/3151917340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2004c3c0 con 0x7f4f20068710 2023-10-07T07:57:33.516 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.511+0000 7f4f077fe700 1 -- 172.21.15.172:0/3151917340 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4f1001bd50 con 0x7f4f20068710 2023-10-07T07:57:33.594 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.593+0000 7f4f25ae4700 1 -- 172.21.15.172:0/3151917340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "26aad281-717c-4bde-8dca-873b8714ac4e", "id": 1} v 0) v1 -- 0x7f4f2006e4d0 con 0x7f4f20068710 2023-10-07T07:57:33.598 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.596+0000 7f4f077fe700 1 -- 172.21.15.172:0/3151917340 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "26aad281-717c-4bde-8dca-873b8714ac4e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4f1002f690 con 0x7f4f20068710 2023-10-07T07:57:33.598 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-10-07T07:57:33.599 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.597+0000 7f4f25ae4700 1 -- 172.21.15.172:0/3151917340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20068710 msgr2=0x7f4f2014f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.599 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.597+0000 7f4f25ae4700 1 --2- 172.21.15.172:0/3151917340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20068710 0x7f4f2014f8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4f10015000 tx=0x7f4f10016000 comp rx=0 tx=0).stop 2023-10-07T07:57:33.599 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.597+0000 7f4f25ae4700 1 -- 172.21.15.172:0/3151917340 shutdown_connections 2023-10-07T07:57:33.599 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.597+0000 7f4f25ae4700 1 --2- 172.21.15.172:0/3151917340 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2015ccf0 0x7f4f200c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.599 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.597+0000 7f4f25ae4700 1 --2- 172.21.15.172:0/3151917340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20068710 0x7f4f2014f8e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.600 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.597+0000 7f4f25ae4700 1 --2- 172.21.15.172:0/3151917340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f2015c920 0x7f4f200b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.600 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.597+0000 7f4f25ae4700 1 -- 172.21.15.172:0/3151917340 >> 172.21.15.172:0/3151917340 conn(0x7f4f200c9c30 msgr2=0x7f4f2006dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:33.600 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.598+0000 7f4f25ae4700 1 -- 172.21.15.172:0/3151917340 shutdown_connections 2023-10-07T07:57:33.600 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.598+0000 7f4f25ae4700 1 -- 172.21.15.172:0/3151917340 wait complete. 2023-10-07T07:57:33.610 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 7b622d63-3165-4b67-8545-fb0d62cc728b 2 2023-10-07T07:57:33.670 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:33.669+0000 7f8a905001c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-07T07:57:33.731 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.728+0000 7fdf501ac700 1 Processor -- start 2023-10-07T07:57:33.731 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.728+0000 7fdf501ac700 1 -- start start 2023-10-07T07:57:33.731 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf501ac700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf48068000 0x7fdf480683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.731 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf501ac700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf481576b0 0x7fdf4806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.731 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf501ac700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf480ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.731 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf501ac700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf480cc0c0 con 0x7fdf481576b0 2023-10-07T07:57:33.731 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf501ac700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf48068910 con 0x7fdf48068000 2023-10-07T07:57:33.731 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf501ac700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480cb4b0 con 0x7fdf4806a650 2023-10-07T07:57:33.731 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf4df48700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf48068000 0x7fdf480683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.732 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf4e749700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf480ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.732 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf4df48700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf48068000 0x7fdf480683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53704/0 (socket says 172.21.15.172:53704) 2023-10-07T07:57:33.732 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf4df48700 1 -- 172.21.15.172:0/2053218824 learned_addr learned my addr 172.21.15.172:0/2053218824 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:33.733 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf4e749700 1 -- 172.21.15.172:0/2053218824 >> v2:172.21.15.172:3300/0 conn(0x7fdf48068000 msgr2=0x7fdf480683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf4e749700 1 --2- 172.21.15.172:0/2053218824 >> v2:172.21.15.172:3300/0 conn(0x7fdf48068000 0x7fdf480683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf4e749700 1 -- 172.21.15.172:0/2053218824 >> v2:172.21.15.7:3300/0 conn(0x7fdf481576b0 msgr2=0x7fdf4806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.729+0000 7fdf4e749700 1 --2- 172.21.15.172:0/2053218824 >> v2:172.21.15.7:3300/0 conn(0x7fdf481576b0 0x7fdf4806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf4e749700 1 -- 172.21.15.172:0/2053218824 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480cb790 con 0x7fdf4806a650 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf4df48700 1 --2- 172.21.15.172:0/2053218824 >> v2:172.21.15.172:3300/0 conn(0x7fdf48068000 0x7fdf480683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf4e749700 1 --2- 172.21.15.172:0/2053218824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf480ca300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf48069510 tx=0x7fdf40009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bacd2996fb9cae9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf4cf46700 1 -- 172.21.15.172:0/2053218824 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000e070 con 0x7fdf4806a650 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf4cf46700 1 -- 172.21.15.172:0/2053218824 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4000bb60 con 0x7fdf4806a650 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf4cf46700 1 -- 172.21.15.172:0/2053218824 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40012430 con 0x7fdf4806a650 2023-10-07T07:57:33.734 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf501ac700 1 -- 172.21.15.172:0/2053218824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 msgr2=0x7fdf480ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf501ac700 1 --2- 172.21.15.172:0/2053218824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf480ca300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf48069510 tx=0x7fdf40009780 comp rx=0 tx=0).stop 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf501ac700 1 -- 172.21.15.172:0/2053218824 shutdown_connections 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf501ac700 1 --2- 172.21.15.172:0/2053218824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf480ca300 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf501ac700 1 --2- 172.21.15.172:0/2053218824 >> v2:172.21.15.172:3300/0 conn(0x7fdf48068000 0x7fdf480683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf501ac700 1 --2- 172.21.15.172:0/2053218824 >> v2:172.21.15.7:3300/0 conn(0x7fdf481576b0 0x7fdf4806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.730+0000 7fdf501ac700 1 -- 172.21.15.172:0/2053218824 >> 172.21.15.172:0/2053218824 conn(0x7fdf480c9c30 msgr2=0x7fdf4806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.731+0000 7fdf501ac700 1 -- 172.21.15.172:0/2053218824 shutdown_connections 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.731+0000 7fdf501ac700 1 -- 172.21.15.172:0/2053218824 wait complete. 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf501ac700 1 Processor -- start 2023-10-07T07:57:33.735 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf501ac700 1 -- start start 2023-10-07T07:57:33.737 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf501ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf48068000 0x7fdf48157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.737 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf501ac700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf481680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.737 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf501ac700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf481576b0 0x7fdf48148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.737 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf501ac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481685f0 con 0x7fdf48068000 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf501ac700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4815b840 con 0x7fdf481576b0 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf501ac700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48168be0 con 0x7fdf4806a650 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf4e749700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf481576b0 0x7fdf48148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.732+0000 7fdf4d747700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf481680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4e749700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf481576b0 0x7fdf48148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53720/0 (socket says 172.21.15.172:53720) 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4d747700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf481680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:40164/0 (socket says 172.21.15.172:40164) 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4e749700 1 -- 172.21.15.172:0/1527606890 learned_addr learned my addr 172.21.15.172:0/1527606890 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4df48700 1 --2- 172.21.15.172:0/1527606890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf48068000 0x7fdf48157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4d747700 1 -- 172.21.15.172:0/1527606890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf481576b0 msgr2=0x7fdf48148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4d747700 1 --2- 172.21.15.172:0/1527606890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf481576b0 0x7fdf48148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4d747700 1 -- 172.21.15.172:0/1527606890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf48068000 msgr2=0x7fdf48157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.738 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4d747700 1 --2- 172.21.15.172:0/1527606890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf48068000 0x7fdf48157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4d747700 1 -- 172.21.15.172:0/1527606890 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38009480 con 0x7fdf4806a650 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf4d747700 1 --2- 172.21.15.172:0/1527606890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf481680b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b5b0 tx=0x7fdf440090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf3dffb700 1 -- 172.21.15.172:0/1527606890 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400f510 con 0x7fdf4806a650 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf3dffb700 1 -- 172.21.15.172:0/1527606890 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4400f670 con 0x7fdf4806a650 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf501ac700 1 -- 172.21.15.172:0/1527606890 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf40009a20 con 0x7fdf4806a650 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf3dffb700 1 -- 172.21.15.172:0/1527606890 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400fa30 con 0x7fdf4806a650 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf3dffb700 1 -- 172.21.15.172:0/1527606890 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fdf4401a680 con 0x7fdf4806a650 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf501ac700 1 -- 172.21.15.172:0/1527606890 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf481691a0 con 0x7fdf4806a650 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.733+0000 7fdf3dffb700 1 -- 172.21.15.172:0/1527606890 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fdf4400fe30 con 0x7fdf4806a650 2023-10-07T07:57:33.739 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.734+0000 7fdf501ac700 1 -- 172.21.15.172:0/1527606890 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4804c3c0 con 0x7fdf4806a650 2023-10-07T07:57:33.740 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.736+0000 7fdf3dffb700 1 -- 172.21.15.172:0/1527606890 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fdf4401aac0 con 0x7fdf4806a650 2023-10-07T07:57:33.771 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:33.770+0000 7f9216a671c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-07T07:57:33.815 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:33.814+0000 7f8a905001c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-07T07:57:33.819 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.819+0000 7fdf501ac700 1 -- 172.21.15.172:0/1527606890 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7b622d63-3165-4b67-8545-fb0d62cc728b", "id": 2} v 0) v1 -- 0x7fdf480cba70 con 0x7fdf4806a650 2023-10-07T07:57:33.823 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.822+0000 7fdf3dffb700 1 -- 172.21.15.172:0/1527606890 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b622d63-3165-4b67-8545-fb0d62cc728b", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdf4401ad40 con 0x7fdf4806a650 2023-10-07T07:57:33.823 INFO:teuthology.orchestra.run.smithi172.stdout:2 2023-10-07T07:57:33.824 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.823+0000 7fdf501ac700 1 -- 172.21.15.172:0/1527606890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 msgr2=0x7fdf481680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.824 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.823+0000 7fdf501ac700 1 --2- 172.21.15.172:0/1527606890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf481680b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b5b0 tx=0x7fdf440090f0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.824 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.823+0000 7fdf501ac700 1 -- 172.21.15.172:0/1527606890 shutdown_connections 2023-10-07T07:57:33.824 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.823+0000 7fdf501ac700 1 --2- 172.21.15.172:0/1527606890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf481576b0 0x7fdf48148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.824 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.823+0000 7fdf501ac700 1 --2- 172.21.15.172:0/1527606890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf48068000 0x7fdf48157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.825 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.823+0000 7fdf501ac700 1 --2- 172.21.15.172:0/1527606890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf4806a650 0x7fdf481680b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.825 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.823+0000 7fdf501ac700 1 -- 172.21.15.172:0/1527606890 >> 172.21.15.172:0/1527606890 conn(0x7fdf480c9c30 msgr2=0x7fdf480c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:33.825 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.823+0000 7fdf501ac700 1 -- 172.21.15.172:0/1527606890 shutdown_connections 2023-10-07T07:57:33.825 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.823+0000 7fdf501ac700 1 -- 172.21.15.172:0/1527606890 wait complete. 2023-10-07T07:57:33.836 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 6998b22c-4c0e-432c-89fb-c1f8311be875 3 2023-10-07T07:57:33.926 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:33.924+0000 7f9216a671c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-07T07:57:33.957 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.955+0000 7f427f3e6700 1 Processor -- start 2023-10-07T07:57:33.957 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.955+0000 7f427f3e6700 1 -- start start 2023-10-07T07:57:33.959 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427f3e6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42780b39b0 0x7f4278068100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427f3e6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42780b0190 0x7f427815d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427f3e6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4278068640 con 0x7f42780b14a0 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427f3e6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42780b1130 con 0x7f42780b39b0 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427f3e6700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427815a8e0 con 0x7f42780b0190 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d983700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42780b0190 0x7f427815d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d983700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42780b0190 0x7f427815d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53722/0 (socket says 172.21.15.172:53722) 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d983700 1 -- 172.21.15.172:0/1333760979 learned_addr learned my addr 172.21.15.172:0/1333760979 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d182700 1 --2- 172.21.15.172:0/1333760979 >> v2:172.21.15.7:3300/0 conn(0x7f42780b39b0 0x7f4278068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d983700 1 -- 172.21.15.172:0/1333760979 >> v2:172.21.15.7:3300/0 conn(0x7f42780b39b0 msgr2=0x7f4278068100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d983700 1 --2- 172.21.15.172:0/1333760979 >> v2:172.21.15.7:3300/0 conn(0x7f42780b39b0 0x7f4278068100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d983700 1 -- 172.21.15.172:0/1333760979 >> v1:172.21.15.7:6789/0 conn(0x7f42780b14a0 legacy=0x7f42780afa80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:33.960 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d983700 1 -- 172.21.15.172:0/1333760979 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4278159cd0 con 0x7f42780b0190 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d182700 1 --2- 172.21.15.172:0/1333760979 >> v2:172.21.15.7:3300/0 conn(0x7f42780b39b0 0x7f4278068100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f427d983700 1 --2- 172.21.15.172:0/1333760979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 0x7f427815d260 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f427806def0 tx=0x7f4270009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a77eb4fd41408121 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f426ffff700 1 -- 172.21.15.172:0/1333760979 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000e070 con 0x7f42780b0190 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f426ffff700 1 -- 172.21.15.172:0/1333760979 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427000ba10 con 0x7f42780b0190 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.956+0000 7f426ffff700 1 -- 172.21.15.172:0/1333760979 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42700092e0 con 0x7f42780b0190 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.957+0000 7f427f3e6700 1 -- 172.21.15.172:0/1333760979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 msgr2=0x7f427815d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.957+0000 7f427f3e6700 1 --2- 172.21.15.172:0/1333760979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 0x7f427815d260 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f427806def0 tx=0x7f4270009840 comp rx=0 tx=0).stop 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.957+0000 7f427f3e6700 1 -- 172.21.15.172:0/1333760979 shutdown_connections 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.957+0000 7f427f3e6700 1 --2- 172.21.15.172:0/1333760979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 0x7f427815d260 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.957+0000 7f427f3e6700 1 --2- 172.21.15.172:0/1333760979 >> v2:172.21.15.7:3300/0 conn(0x7f42780b39b0 0x7f4278068100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.957+0000 7f427f3e6700 1 -- 172.21.15.172:0/1333760979 >> 172.21.15.172:0/1333760979 conn(0x7f427806d350 msgr2=0x7f427806d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:33.961 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.957+0000 7f427f3e6700 1 -- 172.21.15.172:0/1333760979 shutdown_connections 2023-10-07T07:57:33.962 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.957+0000 7f427f3e6700 1 -- 172.21.15.172:0/1333760979 wait complete. 2023-10-07T07:57:33.964 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.959+0000 7f427f3e6700 1 Processor -- start 2023-10-07T07:57:33.964 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.959+0000 7f427f3e6700 1 -- start start 2023-10-07T07:57:33.964 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427f3e6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 0x7f427815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.964 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427f3e6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42780b14a0 0x7f42780a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.964 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427f3e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42780b39b0 0x7f427816c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:33.964 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427f3e6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780a57b0 con 0x7f42780b39b0 2023-10-07T07:57:33.964 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427f3e6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780a61e0 con 0x7f42780b0190 2023-10-07T07:57:33.964 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427f3e6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42780a5bd0 con 0x7f42780b14a0 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427d182700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 0x7f427815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427d182700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 0x7f427815bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53724/0 (socket says 172.21.15.172:53724) 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427d182700 1 -- 172.21.15.172:0/3629211799 learned_addr learned my addr 172.21.15.172:0/3629211799 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427c981700 1 --2- 172.21.15.172:0/3629211799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42780b14a0 0x7f42780a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427c981700 1 -- 172.21.15.172:0/3629211799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 msgr2=0x7f427815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427c981700 1 --2- 172.21.15.172:0/3629211799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 0x7f427815bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427c981700 1 -- 172.21.15.172:0/3629211799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42780b39b0 msgr2=0x7f427816c580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427c981700 1 --2- 172.21.15.172:0/3629211799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42780b39b0 0x7f427816c580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427c981700 1 -- 172.21.15.172:0/3629211799 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270009ae0 con 0x7f42780b14a0 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427d182700 1 --2- 172.21.15.172:0/3629211799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 0x7f427815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f427c981700 1 --2- 172.21.15.172:0/3629211799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42780b14a0 0x7f42780a6670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f426800b5b0 tx=0x7f42680090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.960+0000 7f426d7fa700 1 -- 172.21.15.172:0/3629211799 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426800f510 con 0x7f42780b14a0 2023-10-07T07:57:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.961+0000 7f426d7fa700 1 -- 172.21.15.172:0/3629211799 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426800f690 con 0x7f42780b14a0 2023-10-07T07:57:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.961+0000 7f426d7fa700 1 -- 172.21.15.172:0/3629211799 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426800fa50 con 0x7f42780b14a0 2023-10-07T07:57:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.961+0000 7f427f3e6700 1 -- 172.21.15.172:0/3629211799 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427816d160 con 0x7f42780b14a0 2023-10-07T07:57:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.961+0000 7f427f3e6700 1 -- 172.21.15.172:0/3629211799 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427816d440 con 0x7f42780b14a0 2023-10-07T07:57:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.961+0000 7f426d7fa700 1 -- 172.21.15.172:0/3629211799 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f426801a400 con 0x7f42780b14a0 2023-10-07T07:57:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.961+0000 7f426d7fa700 1 -- 172.21.15.172:0/3629211799 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f426800fdc0 con 0x7f42780b14a0 2023-10-07T07:57:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.962+0000 7f427f3e6700 1 -- 172.21.15.172:0/3629211799 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42780a6350 con 0x7f42780b14a0 2023-10-07T07:57:33.967 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:33.964+0000 7f426d7fa700 1 -- 172.21.15.172:0/3629211799 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f426801a6b0 con 0x7f42780b14a0 2023-10-07T07:57:34.048 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.047+0000 7f427f3e6700 1 -- 172.21.15.172:0/3629211799 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6998b22c-4c0e-432c-89fb-c1f8311be875", "id": 3} v 0) v1 -- 0x7f427804c3c0 con 0x7f42780b14a0 2023-10-07T07:57:34.051 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.050+0000 7f426d7fa700 1 -- 172.21.15.172:0/3629211799 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6998b22c-4c0e-432c-89fb-c1f8311be875", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f42680290b0 con 0x7f42780b14a0 2023-10-07T07:57:34.052 INFO:teuthology.orchestra.run.smithi172.stdout:3 2023-10-07T07:57:34.053 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.051+0000 7f427f3e6700 1 -- 172.21.15.172:0/3629211799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42780b14a0 msgr2=0x7f42780a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.053 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.051+0000 7f427f3e6700 1 --2- 172.21.15.172:0/3629211799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42780b14a0 0x7f42780a6670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f426800b5b0 tx=0x7f42680090f0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.053 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.051+0000 7f427f3e6700 1 -- 172.21.15.172:0/3629211799 shutdown_connections 2023-10-07T07:57:34.053 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.051+0000 7f427f3e6700 1 --2- 172.21.15.172:0/3629211799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42780b39b0 0x7f427816c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.053 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.051+0000 7f427f3e6700 1 --2- 172.21.15.172:0/3629211799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42780b0190 0x7f427815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.053 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.051+0000 7f427f3e6700 1 --2- 172.21.15.172:0/3629211799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42780b14a0 0x7f42780a6670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.053 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.051+0000 7f427f3e6700 1 -- 172.21.15.172:0/3629211799 >> 172.21.15.172:0/3629211799 conn(0x7f427806d350 msgr2=0x7f4278158ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:34.054 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.051+0000 7f427f3e6700 1 -- 172.21.15.172:0/3629211799 shutdown_connections 2023-10-07T07:57:34.054 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.051+0000 7f427f3e6700 1 -- 172.21.15.172:0/3629211799 wait complete. 2023-10-07T07:57:34.064 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 8347d98c-5559-43e5-83e2-e053778e801f 4 2023-10-07T07:57:34.172 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:34.171+0000 7f8a905001c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-07T07:57:34.183 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.181+0000 7fa985adf700 1 Processor -- start 2023-10-07T07:57:34.184 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.181+0000 7fa985adf700 1 -- start start 2023-10-07T07:57:34.186 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.182+0000 7fa985adf700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa98015ccf0 0x7fa9800ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.186 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.182+0000 7fa985adf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9800cab00 con 0x7fa980068710 2023-10-07T07:57:34.186 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.182+0000 7fa985adf700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98006a7a0 con 0x7fa98015c920 2023-10-07T07:57:34.186 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.182+0000 7fa985adf700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9800cb3c0 con 0x7fa98015ccf0 2023-10-07T07:57:34.186 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.182+0000 7fa97ffff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa98015ccf0 0x7fa9800ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.186 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.182+0000 7fa97f7fe700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa98015c920 0x7fa980068000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54354/0 (socket says 172.21.15.172:54354) 2023-10-07T07:57:34.186 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.182+0000 7fa97f7fe700 1 -- 172.21.15.172:0/3418636379 learned_addr learned my addr 172.21.15.172:0/3418636379 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.183+0000 7fa97e7fc700 1 -- 172.21.15.172:0/3418636379 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307020775 0 0) 0x7fa98006a7a0 con 0x7fa98015c920 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.183+0000 7fa97e7fc700 1 -- 172.21.15.172:0/3418636379 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa968003570 con 0x7fa98015c920 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.183+0000 7fa97ffff700 1 -- 172.21.15.172:0/3418636379 >> v1:172.21.15.172:6789/0 conn(0x7fa98015c920 legacy=0x7fa980068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.183+0000 7fa97ffff700 1 -- 172.21.15.172:0/3418636379 >> v1:172.21.15.7:6789/0 conn(0x7fa980068710 legacy=0x7fa98006ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.183+0000 7fa97ffff700 1 -- 172.21.15.172:0/3418636379 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800cae00 con 0x7fa98015ccf0 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.183+0000 7fa97ffff700 1 --2- 172.21.15.172:0/3418636379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015ccf0 0x7fa9800ca520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9780098e0 tx=0x7fa9780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aed91d51a4e2e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.183+0000 7fa97e7fc700 1 -- 172.21.15.172:0/3418636379 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97800b640 con 0x7fa98015ccf0 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.183+0000 7fa97e7fc700 1 -- 172.21.15.172:0/3418636379 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97800b7a0 con 0x7fa98015ccf0 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.183+0000 7fa97e7fc700 1 -- 172.21.15.172:0/3418636379 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97800bb40 con 0x7fa98015ccf0 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.184+0000 7fa985adf700 1 -- 172.21.15.172:0/3418636379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015ccf0 msgr2=0x7fa9800ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.184+0000 7fa985adf700 1 --2- 172.21.15.172:0/3418636379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015ccf0 0x7fa9800ca520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9780098e0 tx=0x7fa9780091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.187 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.184+0000 7fa985adf700 1 -- 172.21.15.172:0/3418636379 shutdown_connections 2023-10-07T07:57:34.188 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.184+0000 7fa985adf700 1 --2- 172.21.15.172:0/3418636379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015ccf0 0x7fa9800ca520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.188 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.184+0000 7fa985adf700 1 -- 172.21.15.172:0/3418636379 >> 172.21.15.172:0/3418636379 conn(0x7fa9800c9c30 msgr2=0x7fa98006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:34.188 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.184+0000 7fa985adf700 1 -- 172.21.15.172:0/3418636379 shutdown_connections 2023-10-07T07:57:34.188 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.184+0000 7fa985adf700 1 -- 172.21.15.172:0/3418636379 wait complete. 2023-10-07T07:57:34.190 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.185+0000 7fa985adf700 1 Processor -- start 2023-10-07T07:57:34.190 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.185+0000 7fa985adf700 1 -- start start 2023-10-07T07:57:34.190 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa985adf700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa980068710 0x7fa980140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.191 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa985adf700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015c920 0x7fa9800af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.191 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa985adf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98015ccf0 0x7fa9800b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.191 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa985adf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa980154ab0 con 0x7fa98015ccf0 2023-10-07T07:57:34.191 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa985adf700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800c4ac0 con 0x7fa98015c920 2023-10-07T07:57:34.191 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa985adf700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800c3eb0 con 0x7fa980068710 2023-10-07T07:57:34.191 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97effd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015c920 0x7fa9800af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.191 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97f7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa980068710 0x7fa980140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.191 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97effd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015c920 0x7fa9800af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53742/0 (socket says 172.21.15.172:53742) 2023-10-07T07:57:34.191 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97effd700 1 -- 172.21.15.172:0/2813843924 learned_addr learned my addr 172.21.15.172:0/2813843924 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97f7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa980068710 0x7fa980140f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:40174/0 (socket says 172.21.15.172:40174) 2023-10-07T07:57:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97f7fe700 1 -- 172.21.15.172:0/2813843924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015c920 msgr2=0x7fa9800af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97f7fe700 1 --2- 172.21.15.172:0/2813843924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015c920 0x7fa9800af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97f7fe700 1 -- 172.21.15.172:0/2813843924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98015ccf0 msgr2=0x7fa9800b1190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-07T07:57:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97f7fe700 1 --2- 172.21.15.172:0/2813843924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98015ccf0 0x7fa9800b1190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97f7fe700 1 -- 172.21.15.172:0/2813843924 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa978009420 con 0x7fa980068710 2023-10-07T07:57:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97effd700 1 --2- 172.21.15.172:0/2813843924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015c920 0x7fa9800af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa97f7fe700 1 --2- 172.21.15.172:0/2813843924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa980068710 0x7fa980140f50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9700016e0 tx=0x7fa970007f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa9677fe700 1 -- 172.21.15.172:0/2813843924 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970005d80 con 0x7fa980068710 2023-10-07T07:57:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa9677fe700 1 -- 172.21.15.172:0/2813843924 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9700070f0 con 0x7fa980068710 2023-10-07T07:57:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa985adf700 1 -- 172.21.15.172:0/2813843924 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9800c4190 con 0x7fa980068710 2023-10-07T07:57:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.186+0000 7fa9677fe700 1 -- 172.21.15.172:0/2813843924 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9700074b0 con 0x7fa980068710 2023-10-07T07:57:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.187+0000 7fa985adf700 1 -- 172.21.15.172:0/2813843924 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9800c4750 con 0x7fa980068710 2023-10-07T07:57:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.187+0000 7fa9677fe700 1 -- 172.21.15.172:0/2813843924 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa9700078e0 con 0x7fa980068710 2023-10-07T07:57:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.187+0000 7fa9677fe700 1 -- 172.21.15.172:0/2813843924 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fa970012400 con 0x7fa980068710 2023-10-07T07:57:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.187+0000 7fa985adf700 1 -- 172.21.15.172:0/2813843924 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98004c3c0 con 0x7fa980068710 2023-10-07T07:57:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.191+0000 7fa9677fe700 1 -- 172.21.15.172:0/2813843924 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa97001b400 con 0x7fa980068710 2023-10-07T07:57:34.276 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.274+0000 7fa985adf700 1 -- 172.21.15.172:0/2813843924 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8347d98c-5559-43e5-83e2-e053778e801f", "id": 4} v 0) v1 -- 0x7fa980069540 con 0x7fa980068710 2023-10-07T07:57:34.346 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:34.304+0000 7f9216a671c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-07T07:57:34.346 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.278+0000 7fa9677fe700 1 -- 172.21.15.172:0/2813843924 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8347d98c-5559-43e5-83e2-e053778e801f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa97001b5a0 con 0x7fa980068710 2023-10-07T07:57:34.346 INFO:teuthology.orchestra.run.smithi172.stdout:4 2023-10-07T07:57:34.348 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.279+0000 7fa985adf700 1 -- 172.21.15.172:0/2813843924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa980068710 msgr2=0x7fa980140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.348 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.279+0000 7fa985adf700 1 --2- 172.21.15.172:0/2813843924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa980068710 0x7fa980140f50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9700016e0 tx=0x7fa970007f40 comp rx=0 tx=0).stop 2023-10-07T07:57:34.348 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.279+0000 7fa985adf700 1 -- 172.21.15.172:0/2813843924 shutdown_connections 2023-10-07T07:57:34.348 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.279+0000 7fa985adf700 1 --2- 172.21.15.172:0/2813843924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa98015ccf0 0x7fa9800b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.348 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.279+0000 7fa985adf700 1 --2- 172.21.15.172:0/2813843924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa980068710 0x7fa980140f50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.348 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.279+0000 7fa985adf700 1 --2- 172.21.15.172:0/2813843924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa98015c920 0x7fa9800af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.348 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.279+0000 7fa985adf700 1 -- 172.21.15.172:0/2813843924 >> 172.21.15.172:0/2813843924 conn(0x7fa9800c9c30 msgr2=0x7fa980069bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:34.348 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.279+0000 7fa985adf700 1 -- 172.21.15.172:0/2813843924 shutdown_connections 2023-10-07T07:57:34.348 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.279+0000 7fa985adf700 1 -- 172.21.15.172:0/2813843924 wait complete. 2023-10-07T07:57:34.349 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 189820da-0b58-4f63-9405-647998088464 5 2023-10-07T07:57:34.468 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.466+0000 7f083c92d700 1 Processor -- start 2023-10-07T07:57:34.468 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.466+0000 7f083c92d700 1 -- start start 2023-10-07T07:57:34.470 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.467+0000 7f083c92d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f083815a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.470 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.467+0000 7f083c92d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f08380ccc90 0x7f08380cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.470 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.467+0000 7f083c92d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08380ce480 con 0x7f08380af860 2023-10-07T07:57:34.470 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.467+0000 7f083c92d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08380ce760 con 0x7f08380ccc90 2023-10-07T07:57:34.470 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.467+0000 7f083c92d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08380cd610 con 0x7f08380af490 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f083659c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f083815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f0836d9d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f08380ccc90 0x7f08380cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f0836d9d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f08380ccc90 0x7f08380cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53750/0 (socket says 172.21.15.172:53750) 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f083659c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f083815a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:40182/0 (socket says 172.21.15.172:40182) 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f0836d9d700 1 -- 172.21.15.172:0/528677916 learned_addr learned my addr 172.21.15.172:0/528677916 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f0836d9d700 1 -- 172.21.15.172:0/528677916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 msgr2=0x7f083815a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f0836d9d700 1 --2- 172.21.15.172:0/528677916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f083815a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f0836d9d700 1 -- 172.21.15.172:0/528677916 >> v1:172.21.15.7:6789/0 conn(0x7f08380af860 legacy=0x7f083815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f0836d9d700 1 -- 172.21.15.172:0/528677916 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083815d720 con 0x7f08380ccc90 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f0836d9d700 1 --2- 172.21.15.172:0/528677916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 0x7f08380cd0d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0830009f20 tx=0x7f0830009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=302dbd7d5531788 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:34.471 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f083559a700 1 -- 172.21.15.172:0/528677916 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083000e070 con 0x7f08380ccc90 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f083559a700 1 -- 172.21.15.172:0/528677916 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083000b9d0 con 0x7f08380ccc90 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.468+0000 7f083559a700 1 -- 172.21.15.172:0/528677916 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08300092e0 con 0x7f08380ccc90 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.469+0000 7f083c92d700 1 -- 172.21.15.172:0/528677916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 msgr2=0x7f08380cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.469+0000 7f083c92d700 1 --2- 172.21.15.172:0/528677916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 0x7f08380cd0d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0830009f20 tx=0x7f0830009840 comp rx=0 tx=0).stop 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.469+0000 7f083c92d700 1 -- 172.21.15.172:0/528677916 shutdown_connections 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.469+0000 7f083c92d700 1 --2- 172.21.15.172:0/528677916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 0x7f08380cd0d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.469+0000 7f083c92d700 1 --2- 172.21.15.172:0/528677916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f083815a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.469+0000 7f083c92d700 1 -- 172.21.15.172:0/528677916 >> 172.21.15.172:0/528677916 conn(0x7f0838067be0 msgr2=0x7f0838067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.469+0000 7f083c92d700 1 -- 172.21.15.172:0/528677916 shutdown_connections 2023-10-07T07:57:34.472 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.469+0000 7f083c92d700 1 -- 172.21.15.172:0/528677916 wait complete. 2023-10-07T07:57:34.474 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.470+0000 7f083c92d700 1 Processor -- start 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.470+0000 7f083c92d700 1 -- start start 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083c92d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f0838148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083c92d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380af860 0x7f083816d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083c92d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 0x7f08380ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083c92d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08380b5510 con 0x7f08380af860 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083c92d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083816dbb0 con 0x7f08380ccc90 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083c92d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083816d1e0 con 0x7f08380af490 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083659c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f0838148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0836d9d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 0x7f08380ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0836d9d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 0x7f08380ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53758/0 (socket says 172.21.15.172:53758) 2023-10-07T07:57:34.475 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0836d9d700 1 -- 172.21.15.172:0/3270503307 learned_addr learned my addr 172.21.15.172:0/3270503307 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083659c700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f0838148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:40184/0 (socket says 172.21.15.172:40184) 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0835d9b700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380af860 0x7f083816d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0836d9d700 1 -- 172.21.15.172:0/3270503307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 msgr2=0x7f0838148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0836d9d700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f0838148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0836d9d700 1 -- 172.21.15.172:0/3270503307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380af860 msgr2=0x7f083816d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0836d9d700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380af860 0x7f083816d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0836d9d700 1 -- 172.21.15.172:0/3270503307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0830009ae0 con 0x7f08380ccc90 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0835d9b700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380af860 0x7f083816d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f0836d9d700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 0x7f08380ca9a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f083000b610 tx=0x7f0830018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f08267fc700 1 -- 172.21.15.172:0/3270503307 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830008b80 con 0x7f08380ccc90 2023-10-07T07:57:34.476 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f08267fc700 1 -- 172.21.15.172:0/3270503307 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0830008ce0 con 0x7f08380ccc90 2023-10-07T07:57:34.477 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f08267fc700 1 -- 172.21.15.172:0/3270503307 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830025690 con 0x7f08380ccc90 2023-10-07T07:57:34.477 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083c92d700 1 -- 172.21.15.172:0/3270503307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08380b4f50 con 0x7f08380ccc90 2023-10-07T07:57:34.477 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.471+0000 7f083c92d700 1 -- 172.21.15.172:0/3270503307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083816ce70 con 0x7f08380ccc90 2023-10-07T07:57:34.477 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.472+0000 7f08267fc700 1 -- 172.21.15.172:0/3270503307 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0830003040 con 0x7f08380ccc90 2023-10-07T07:57:34.477 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.472+0000 7f08267fc700 1 -- 172.21.15.172:0/3270503307 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f0830025e40 con 0x7f08380ccc90 2023-10-07T07:57:34.477 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.472+0000 7f083c92d700 1 -- 172.21.15.172:0/3270503307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083816ccd0 con 0x7f08380ccc90 2023-10-07T07:57:34.477 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.476+0000 7f08267fc700 1 -- 172.21.15.172:0/3270503307 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f083816ccd0 con 0x7f08380ccc90 2023-10-07T07:57:34.560 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.559+0000 7f083c92d700 1 -- 172.21.15.172:0/3270503307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "189820da-0b58-4f63-9405-647998088464", "id": 5} v 0) v1 -- 0x7f08380716c0 con 0x7f08380ccc90 2023-10-07T07:57:34.564 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.562+0000 7f08267fc700 1 -- 172.21.15.172:0/3270503307 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "189820da-0b58-4f63-9405-647998088464", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0830020070 con 0x7f08380ccc90 2023-10-07T07:57:34.564 INFO:teuthology.orchestra.run.smithi172.stdout:5 2023-10-07T07:57:34.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.563+0000 7f083c92d700 1 -- 172.21.15.172:0/3270503307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 msgr2=0x7f08380ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.563+0000 7f083c92d700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 0x7f08380ca9a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f083000b610 tx=0x7f0830018000 comp rx=0 tx=0).stop 2023-10-07T07:57:34.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.563+0000 7f083c92d700 1 -- 172.21.15.172:0/3270503307 shutdown_connections 2023-10-07T07:57:34.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.563+0000 7f083c92d700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08380ccc90 0x7f08380ca9a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.563+0000 7f083c92d700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08380af490 0x7f0838148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.563+0000 7f083c92d700 1 --2- 172.21.15.172:0/3270503307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380af860 0x7f083816d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.563+0000 7f083c92d700 1 -- 172.21.15.172:0/3270503307 >> 172.21.15.172:0/3270503307 conn(0x7f0838067be0 msgr2=0x7f0838157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:34.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.563+0000 7f083c92d700 1 -- 172.21.15.172:0/3270503307 shutdown_connections 2023-10-07T07:57:34.566 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.563+0000 7f083c92d700 1 -- 172.21.15.172:0/3270503307 wait complete. 2023-10-07T07:57:34.577 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 070138d2-5051-4c4a-bfa3-8afbdcbb0806 6 2023-10-07T07:57:34.696 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.695+0000 7fd9a3e4f700 1 Processor -- start 2023-10-07T07:57:34.696 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.695+0000 7fd9a3e4f700 1 -- start start 2023-10-07T07:57:34.699 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.696+0000 7fd9a3e4f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd99c15c920 0x7fd99c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.699 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.696+0000 7fd9a3e4f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99c0cab00 con 0x7fd99c15ccf0 2023-10-07T07:57:34.699 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.696+0000 7fd9a3e4f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99c06a7a0 con 0x7fd99c068710 2023-10-07T07:57:34.699 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.696+0000 7fd9a3e4f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99c0cb3c0 con 0x7fd99c15c920 2023-10-07T07:57:34.699 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.696+0000 7fd9a23ec700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd99c068710 0x7fd99c06ac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54358/0 (socket says 172.21.15.172:54358) 2023-10-07T07:57:34.699 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.696+0000 7fd9a23ec700 1 -- 172.21.15.172:0/4024750922 learned_addr learned my addr 172.21.15.172:0/4024750922 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-10-07T07:57:34.699 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.696+0000 7fd9a1beb700 1 --2- 172.21.15.172:0/4024750922 >> v2:172.21.15.7:3300/0 conn(0x7fd99c15c920 0x7fd99c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.696+0000 7fd9a0be9700 1 -- 172.21.15.172:0/4024750922 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199626460 0 0) 0x7fd99c06a7a0 con 0x7fd99c068710 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.696+0000 7fd9a0be9700 1 -- 172.21.15.172:0/4024750922 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd988003570 con 0x7fd99c068710 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.697+0000 7fd9a0be9700 1 -- 172.21.15.172:0/4024750922 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389773743 0 0) 0x7fd988003570 con 0x7fd99c068710 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.697+0000 7fd9a1beb700 1 -- 172.21.15.172:0/4024750922 >> v1:172.21.15.172:6789/0 conn(0x7fd99c068710 legacy=0x7fd99c06ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.697+0000 7fd9a1beb700 1 -- 172.21.15.172:0/4024750922 >> v1:172.21.15.7:6789/0 conn(0x7fd99c15ccf0 legacy=0x7fd99c068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.697+0000 7fd9a1beb700 1 -- 172.21.15.172:0/4024750922 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c0cae00 con 0x7fd99c15c920 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.697+0000 7fd9a1beb700 1 --2- 172.21.15.172:0/4024750922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c15c920 0x7fd99c0ca520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009850 tx=0x7fd98c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bad7f646857f0a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.697+0000 7fd9a0be9700 1 -- 172.21.15.172:0/4024750922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9940017f0 con 0x7fd99c15c920 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.697+0000 7fd9a0be9700 1 -- 172.21.15.172:0/4024750922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c011420 con 0x7fd99c15c920 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.697+0000 7fd9a0be9700 1 -- 172.21.15.172:0/4024750922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0117e0 con 0x7fd99c15c920 2023-10-07T07:57:34.700 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.698+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/4024750922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c15c920 msgr2=0x7fd99c0ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.701 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.698+0000 7fd9a3e4f700 1 --2- 172.21.15.172:0/4024750922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c15c920 0x7fd99c0ca520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009850 tx=0x7fd98c009120 comp rx=0 tx=0).stop 2023-10-07T07:57:34.701 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.698+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/4024750922 shutdown_connections 2023-10-07T07:57:34.701 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.698+0000 7fd9a3e4f700 1 --2- 172.21.15.172:0/4024750922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c15c920 0x7fd99c0ca520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.701 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.698+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/4024750922 >> 172.21.15.172:0/4024750922 conn(0x7fd99c0c9c30 msgr2=0x7fd99c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:34.701 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.698+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/4024750922 shutdown_connections 2023-10-07T07:57:34.701 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.698+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/4024750922 wait complete. 2023-10-07T07:57:34.703 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.699+0000 7fd9a3e4f700 1 Processor -- start 2023-10-07T07:57:34.703 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.699+0000 7fd9a3e4f700 1 -- start start 2023-10-07T07:57:34.704 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.699+0000 7fd9a3e4f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd99c068710 0x7fd99c14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.704 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.699+0000 7fd9a3e4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c15c920 0x7fd99c0b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.704 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.699+0000 7fd9a3e4f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd99c15ccf0 0x7fd99c0c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.704 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.699+0000 7fd9a3e4f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c15c040 con 0x7fd99c15c920 2023-10-07T07:57:34.704 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.699+0000 7fd9a3e4f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c15b430 con 0x7fd99c068710 2023-10-07T07:57:34.704 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a3e4f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c15b710 con 0x7fd99c15ccf0 2023-10-07T07:57:34.704 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a1beb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd99c068710 0x7fd99c14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.704 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a23ec700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd99c15ccf0 0x7fd99c0c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a1beb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd99c068710 0x7fd99c14a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53772/0 (socket says 172.21.15.172:53772) 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a1beb700 1 -- 172.21.15.172:0/3519586354 learned_addr learned my addr 172.21.15.172:0/3519586354 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a13ea700 1 --2- 172.21.15.172:0/3519586354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c15c920 0x7fd99c0b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a1beb700 1 -- 172.21.15.172:0/3519586354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd99c15ccf0 msgr2=0x7fd99c0c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a1beb700 1 --2- 172.21.15.172:0/3519586354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd99c15ccf0 0x7fd99c0c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a1beb700 1 -- 172.21.15.172:0/3519586354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c15c920 msgr2=0x7fd99c0b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a1beb700 1 --2- 172.21.15.172:0/3519586354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c15c920 0x7fd99c0b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a1beb700 1 -- 172.21.15.172:0/3519586354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c0093c0 con 0x7fd99c068710 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a1beb700 1 --2- 172.21.15.172:0/3519586354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd99c068710 0x7fd99c14a310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd99c06d500 tx=0x7fd98c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:34.705 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd991ffb700 1 -- 172.21.15.172:0/3519586354 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c000d10 con 0x7fd99c068710 2023-10-07T07:57:34.706 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd991ffb700 1 -- 172.21.15.172:0/3519586354 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c01caf0 con 0x7fd99c068710 2023-10-07T07:57:34.706 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd991ffb700 1 -- 172.21.15.172:0/3519586354 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c01a450 con 0x7fd99c068710 2023-10-07T07:57:34.706 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/3519586354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99c15b9f0 con 0x7fd99c068710 2023-10-07T07:57:34.706 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.700+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/3519586354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c14a850 con 0x7fd99c068710 2023-10-07T07:57:34.706 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.701+0000 7fd991ffb700 1 -- 172.21.15.172:0/3519586354 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd98c01aa80 con 0x7fd99c068710 2023-10-07T07:57:34.706 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.701+0000 7fd991ffb700 1 -- 172.21.15.172:0/3519586354 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fd98c024a10 con 0x7fd99c068710 2023-10-07T07:57:34.706 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.701+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/3519586354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99c04c3c0 con 0x7fd99c068710 2023-10-07T07:57:34.707 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.703+0000 7fd991ffb700 1 -- 172.21.15.172:0/3519586354 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd98c029070 con 0x7fd99c068710 2023-10-07T07:57:34.788 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.786+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/3519586354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "070138d2-5051-4c4a-bfa3-8afbdcbb0806", "id": 6} v 0) v1 -- 0x7fd99c069540 con 0x7fd99c068710 2023-10-07T07:57:34.792 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.790+0000 7fd991ffb700 1 -- 172.21.15.172:0/3519586354 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "070138d2-5051-4c4a-bfa3-8afbdcbb0806", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd98c005a00 con 0x7fd99c068710 2023-10-07T07:57:34.792 INFO:teuthology.orchestra.run.smithi172.stdout:6 2023-10-07T07:57:34.793 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.791+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/3519586354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd99c068710 msgr2=0x7fd99c14a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.793 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.791+0000 7fd9a3e4f700 1 --2- 172.21.15.172:0/3519586354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd99c068710 0x7fd99c14a310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd99c06d500 tx=0x7fd98c016000 comp rx=0 tx=0).stop 2023-10-07T07:57:34.794 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.791+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/3519586354 shutdown_connections 2023-10-07T07:57:34.794 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.791+0000 7fd9a3e4f700 1 --2- 172.21.15.172:0/3519586354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd99c15ccf0 0x7fd99c0c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.794 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.791+0000 7fd9a3e4f700 1 --2- 172.21.15.172:0/3519586354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd99c068710 0x7fd99c14a310 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.794 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.791+0000 7fd9a3e4f700 1 --2- 172.21.15.172:0/3519586354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c15c920 0x7fd99c0b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.794 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.791+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/3519586354 >> 172.21.15.172:0/3519586354 conn(0x7fd99c0c9c30 msgr2=0x7fd99c0c7110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:34.794 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.791+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/3519586354 shutdown_connections 2023-10-07T07:57:34.794 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.791+0000 7fd9a3e4f700 1 -- 172.21.15.172:0/3519586354 wait complete. 2023-10-07T07:57:34.805 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 585b3d12-8bca-4a10-9d87-9bb9a95311b2 7 2023-10-07T07:57:34.926 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.924+0000 7fc0622b8700 1 Processor -- start 2023-10-07T07:57:34.926 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.924+0000 7fc0622b8700 1 -- start start 2023-10-07T07:57:34.928 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.925+0000 7fc0622b8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c068710 0x7fc05c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.925+0000 7fc0622b8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05c0cab00 con 0x7fc05c15c920 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.925+0000 7fc0622b8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05c06a7a0 con 0x7fc05c15ccf0 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.925+0000 7fc0622b8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c0cb3c0 con 0x7fc05c068710 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.925+0000 7fc053fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c068710 0x7fc05c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.925+0000 7fc053fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c068710 0x7fc05c0ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:40206/0 (socket says 172.21.15.172:40206) 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.925+0000 7fc053fff700 1 -- 172.21.15.172:0/2106645407 learned_addr learned my addr 172.21.15.172:0/2106645407 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.925+0000 7fc060855700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc05c15ccf0 0x7fc05c06ac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54372/0 (socket says 172.21.15.172:54372) 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.926+0000 7fc05b7fe700 1 -- 172.21.15.172:0/2106645407 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580092033 0 0) 0x7fc05c06a7a0 con 0x7fc05c15ccf0 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.926+0000 7fc05b7fe700 1 -- 172.21.15.172:0/2106645407 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc048003570 con 0x7fc05c15ccf0 2023-10-07T07:57:34.929 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.926+0000 7fc053fff700 1 -- 172.21.15.172:0/2106645407 >> v1:172.21.15.172:6789/0 conn(0x7fc05c15ccf0 legacy=0x7fc05c06ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.926+0000 7fc053fff700 1 -- 172.21.15.172:0/2106645407 >> v1:172.21.15.7:6789/0 conn(0x7fc05c15c920 legacy=0x7fc05c068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.926+0000 7fc053fff700 1 -- 172.21.15.172:0/2106645407 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c0cae00 con 0x7fc05c068710 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.926+0000 7fc053fff700 1 --2- 172.21.15.172:0/2106645407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c068710 0x7fc05c0ca520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc0440099e0 tx=0x7fc044009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42c8e13e995562ab server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.926+0000 7fc05b7fe700 1 -- 172.21.15.172:0/2106645407 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04400b860 con 0x7fc05c068710 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.926+0000 7fc05b7fe700 1 -- 172.21.15.172:0/2106645407 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04400b9c0 con 0x7fc05c068710 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.926+0000 7fc05b7fe700 1 -- 172.21.15.172:0/2106645407 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04400bd80 con 0x7fc05c068710 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.927+0000 7fc0622b8700 1 -- 172.21.15.172:0/2106645407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c068710 msgr2=0x7fc05c0ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.927+0000 7fc0622b8700 1 --2- 172.21.15.172:0/2106645407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c068710 0x7fc05c0ca520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc0440099e0 tx=0x7fc044009250 comp rx=0 tx=0).stop 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.927+0000 7fc0622b8700 1 -- 172.21.15.172:0/2106645407 shutdown_connections 2023-10-07T07:57:34.930 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.927+0000 7fc0622b8700 1 --2- 172.21.15.172:0/2106645407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c068710 0x7fc05c0ca520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.931 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.927+0000 7fc0622b8700 1 -- 172.21.15.172:0/2106645407 >> 172.21.15.172:0/2106645407 conn(0x7fc05c0c9c30 msgr2=0x7fc05c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:34.931 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.927+0000 7fc0622b8700 1 -- 172.21.15.172:0/2106645407 shutdown_connections 2023-10-07T07:57:34.931 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.927+0000 7fc0622b8700 1 -- 172.21.15.172:0/2106645407 wait complete. 2023-10-07T07:57:34.933 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.928+0000 7fc0622b8700 1 Processor -- start 2023-10-07T07:57:34.933 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.928+0000 7fc0622b8700 1 -- start start 2023-10-07T07:57:34.933 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc0622b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05c068710 0x7fc05c147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.933 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc0622b8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc05c15c920 0x7fc05c0c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.933 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc0622b8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c15ccf0 0x7fc05c0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:34.933 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc0622b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c0c4dd0 con 0x7fc05c068710 2023-10-07T07:57:34.933 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc0622b8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c0c41c0 con 0x7fc05c15c920 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc0622b8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c0c44a0 con 0x7fc05c15ccf0 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc053fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc05c15c920 0x7fc05c0c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc060855700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c15ccf0 0x7fc05c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc053fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc05c15c920 0x7fc05c0c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53782/0 (socket says 172.21.15.172:53782) 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc053fff700 1 -- 172.21.15.172:0/206422891 learned_addr learned my addr 172.21.15.172:0/206422891 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc05bfff700 1 --2- 172.21.15.172:0/206422891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05c068710 0x7fc05c147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc060855700 1 -- 172.21.15.172:0/206422891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc05c15c920 msgr2=0x7fc05c0c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc060855700 1 --2- 172.21.15.172:0/206422891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc05c15c920 0x7fc05c0c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc060855700 1 -- 172.21.15.172:0/206422891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05c068710 msgr2=0x7fc05c147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc060855700 1 --2- 172.21.15.172:0/206422891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05c068710 0x7fc05c147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:34.934 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc060855700 1 -- 172.21.15.172:0/206422891 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440094f0 con 0x7fc05c15ccf0 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc053fff700 1 --2- 172.21.15.172:0/206422891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc05c15c920 0x7fc05c0c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc060855700 1 --2- 172.21.15.172:0/206422891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c15ccf0 0x7fc05c0c7d20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc05400cc70 tx=0x7fc054001aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc058ff9700 1 -- 172.21.15.172:0/206422891 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054005df0 con 0x7fc05c15ccf0 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc058ff9700 1 -- 172.21.15.172:0/206422891 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05400a680 con 0x7fc05c15ccf0 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc058ff9700 1 -- 172.21.15.172:0/206422891 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05400aa00 con 0x7fc05c15ccf0 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc0622b8700 1 -- 172.21.15.172:0/206422891 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c0c4780 con 0x7fc05c15ccf0 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.929+0000 7fc0622b8700 1 -- 172.21.15.172:0/206422891 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c147a50 con 0x7fc05c15ccf0 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.930+0000 7fc058ff9700 1 -- 172.21.15.172:0/206422891 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc0540070f0 con 0x7fc05c15ccf0 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.930+0000 7fc058ff9700 1 -- 172.21.15.172:0/206422891 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fc05400adf0 con 0x7fc05c15ccf0 2023-10-07T07:57:34.935 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.930+0000 7fc0622b8700 1 -- 172.21.15.172:0/206422891 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c0c4f40 con 0x7fc05c15ccf0 2023-10-07T07:57:34.936 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:34.932+0000 7fc058ff9700 1 -- 172.21.15.172:0/206422891 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc054007580 con 0x7fc05c15ccf0 2023-10-07T07:57:34.938 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:34.937+0000 7f8a905001c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-07T07:57:35.012 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:35.010+0000 7f8a905001c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-07T07:57:35.015 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.014+0000 7fc0622b8700 1 -- 172.21.15.172:0/206422891 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "585b3d12-8bca-4a10-9d87-9bb9a95311b2", "id": 7} v 0) v1 -- 0x7fc05c04c3c0 con 0x7fc05c15ccf0 2023-10-07T07:57:35.019 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.018+0000 7fc058ff9700 1 -- 172.21.15.172:0/206422891 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "585b3d12-8bca-4a10-9d87-9bb9a95311b2", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc054007720 con 0x7fc05c15ccf0 2023-10-07T07:57:35.019 INFO:teuthology.orchestra.run.smithi172.stdout:7 2023-10-07T07:57:35.020 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.018+0000 7fc0622b8700 1 -- 172.21.15.172:0/206422891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c15ccf0 msgr2=0x7fc05c0c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:35.020 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.018+0000 7fc0622b8700 1 --2- 172.21.15.172:0/206422891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c15ccf0 0x7fc05c0c7d20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc05400cc70 tx=0x7fc054001aa0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.020 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.019+0000 7fc0622b8700 1 -- 172.21.15.172:0/206422891 shutdown_connections 2023-10-07T07:57:35.020 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.019+0000 7fc0622b8700 1 --2- 172.21.15.172:0/206422891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc05c15ccf0 0x7fc05c0c7d20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.020 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.019+0000 7fc0622b8700 1 --2- 172.21.15.172:0/206422891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05c068710 0x7fc05c147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.021 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.019+0000 7fc0622b8700 1 --2- 172.21.15.172:0/206422891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc05c15c920 0x7fc05c0c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.021 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.019+0000 7fc0622b8700 1 -- 172.21.15.172:0/206422891 >> 172.21.15.172:0/206422891 conn(0x7fc05c0c9c30 msgr2=0x7fc05c0c1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:35.021 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.019+0000 7fc0622b8700 1 -- 172.21.15.172:0/206422891 shutdown_connections 2023-10-07T07:57:35.021 INFO:teuthology.orchestra.run.smithi172.stderr:2023-10-07T07:57:35.019+0000 7fc0622b8700 1 -- 172.21.15.172:0/206422891 wait complete. 2023-10-07T07:57:35.031 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-07T07:57:35.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-07T07:57:35.034 INFO:tasks.ceph.osd.0:Started 2023-10-07T07:57:35.034 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-07T07:57:35.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-07T07:57:35.036 INFO:tasks.ceph.osd.1:Started 2023-10-07T07:57:35.036 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-07T07:57:35.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-07T07:57:35.038 INFO:tasks.ceph.osd.2:Started 2023-10-07T07:57:35.038 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-07T07:57:35.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-07T07:57:35.040 INFO:tasks.ceph.osd.3:Started 2023-10-07T07:57:35.040 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-07T07:57:35.041 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-07T07:57:35.044 INFO:tasks.ceph.osd.4:Started 2023-10-07T07:57:35.044 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-07T07:57:35.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-07T07:57:35.047 INFO:tasks.ceph.osd.5:Started 2023-10-07T07:57:35.047 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-07T07:57:35.047 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-07T07:57:35.051 INFO:tasks.ceph.osd.6:Started 2023-10-07T07:57:35.051 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-07T07:57:35.051 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-07T07:57:35.054 INFO:tasks.ceph.osd.7:Started 2023-10-07T07:57:35.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:35.168 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:35.166+0000 7f9216a671c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-07T07:57:35.168 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:35.167+0000 7f8a905001c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-07T07:57:35.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.224+0000 7f883f5e9700 1 Processor -- start 2023-10-07T07:57:35.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.224+0000 7f883f5e9700 1 -- start start 2023-10-07T07:57:35.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.225+0000 7f883f5e9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9af0 0x7f883815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:35.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.225+0000 7f883f5e9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883815a330 con 0x7f8838063230 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.225+0000 7f883f5e9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883815a610 con 0x7f88380c9ec0 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.225+0000 7f883f5e9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88380b03a0 con 0x7f88380c9af0 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.225+0000 7f883cb84700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8838063230 0x7f883806db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43890/0 (socket says 172.21.15.7:43890) 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.225+0000 7f883cb84700 1 -- 172.21.15.7:0/2143091870 learned_addr learned my addr 172.21.15.7:0/2143091870 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.225+0000 7f883d385700 1 --2- 172.21.15.7:0/2143091870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9af0 0x7f883815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f8837fff700 1 -- 172.21.15.7:0/2143091870 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586390803 0 0) 0x7f883815a330 con 0x7f8838063230 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f8837fff700 1 -- 172.21.15.7:0/2143091870 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8820003570 con 0x7f8838063230 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f8837fff700 1 -- 172.21.15.7:0/2143091870 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869249296 0 0) 0x7f883815a610 con 0x7f88380c9ec0 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f8837fff700 1 -- 172.21.15.7:0/2143091870 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f883815a330 con 0x7f88380c9ec0 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f883d385700 1 -- 172.21.15.7:0/2143091870 >> v1:172.21.15.172:6789/0 conn(0x7f88380c9ec0 legacy=0x7f8838158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f883d385700 1 -- 172.21.15.7:0/2143091870 >> v1:172.21.15.7:6789/0 conn(0x7f8838063230 legacy=0x7f883806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f883d385700 1 -- 172.21.15.7:0/2143091870 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883815a8f0 con 0x7f88380c9af0 2023-10-07T07:57:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f883d385700 1 --2- 172.21.15.7:0/2143091870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9af0 0x7f883815d8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8828009e90 tx=0x7f8828009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4781fdb230a1334 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f8837fff700 1 -- 172.21.15.7:0/2143091870 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800e070 con 0x7f88380c9af0 2023-10-07T07:57:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f8837fff700 1 -- 172.21.15.7:0/2143091870 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882800bb80 con 0x7f88380c9af0 2023-10-07T07:57:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.226+0000 7f8837fff700 1 -- 172.21.15.7:0/2143091870 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828012430 con 0x7f88380c9af0 2023-10-07T07:57:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.227+0000 7f883f5e9700 1 -- 172.21.15.7:0/2143091870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9af0 msgr2=0x7f883815d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.227+0000 7f883f5e9700 1 --2- 172.21.15.7:0/2143091870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9af0 0x7f883815d8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8828009e90 tx=0x7f8828009780 comp rx=0 tx=0).stop 2023-10-07T07:57:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.227+0000 7f883f5e9700 1 -- 172.21.15.7:0/2143091870 shutdown_connections 2023-10-07T07:57:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.227+0000 7f883f5e9700 1 --2- 172.21.15.7:0/2143091870 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9af0 0x7f883815d8f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.227+0000 7f883f5e9700 1 -- 172.21.15.7:0/2143091870 >> 172.21.15.7:0/2143091870 conn(0x7f88380cd1a0 msgr2=0x7f88380c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.227+0000 7f883f5e9700 1 -- 172.21.15.7:0/2143091870 shutdown_connections 2023-10-07T07:57:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.227+0000 7f883f5e9700 1 -- 172.21.15.7:0/2143091870 wait complete. 2023-10-07T07:57:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.228+0000 7f883f5e9700 1 Processor -- start 2023-10-07T07:57:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.228+0000 7f883f5e9700 1 -- start start 2023-10-07T07:57:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.229+0000 7f883f5e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838063230 0x7f88381470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.229+0000 7f883f5e9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88380c9af0 0x7f88380c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.229+0000 7f883f5e9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9ec0 0x7f88380c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.229+0000 7f883f5e9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88380c74f0 con 0x7f8838063230 2023-10-07T07:57:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.229+0000 7f883f5e9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88380a5730 con 0x7f88380c9af0 2023-10-07T07:57:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.229+0000 7f883f5e9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88380c6490 con 0x7f88380c9ec0 2023-10-07T07:57:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.229+0000 7f883d385700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838063230 0x7f88381470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883d385700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838063230 0x7f88381470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40182/0 (socket says 172.21.15.7:40182) 2023-10-07T07:57:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883d385700 1 -- 172.21.15.7:0/74829551 learned_addr learned my addr 172.21.15.7:0/74829551 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883cb84700 1 --2- 172.21.15.7:0/74829551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88380c9af0 0x7f88380c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883d385700 1 -- 172.21.15.7:0/74829551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9ec0 msgr2=0x7f88380c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883d385700 1 --2- 172.21.15.7:0/74829551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9ec0 0x7f88380c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883d385700 1 -- 172.21.15.7:0/74829551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88380c9af0 msgr2=0x7f88380c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883d385700 1 --2- 172.21.15.7:0/74829551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88380c9af0 0x7f88380c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883d385700 1 -- 172.21.15.7:0/74829551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8828009a20 con 0x7f8838063230 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883cb84700 1 --2- 172.21.15.7:0/74829551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88380c9af0 0x7f88380c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883d385700 1 --2- 172.21.15.7:0/74829551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838063230 0x7f88381470f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88280097c0 tx=0x7f8828019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f88357fa700 1 -- 172.21.15.7:0/74829551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800be40 con 0x7f8838063230 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f88357fa700 1 -- 172.21.15.7:0/74829551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8828027470 con 0x7f8838063230 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f88357fa700 1 -- 172.21.15.7:0/74829551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828027830 con 0x7f8838063230 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883f5e9700 1 -- 172.21.15.7:0/74829551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88380c6770 con 0x7f8838063230 2023-10-07T07:57:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.230+0000 7f883f5e9700 1 -- 172.21.15.7:0/74829551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8838147630 con 0x7f8838063230 2023-10-07T07:57:35.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.231+0000 7f88357fa700 1 -- 172.21.15.7:0/74829551 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8828024070 con 0x7f8838063230 2023-10-07T07:57:35.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.231+0000 7f88357fa700 1 -- 172.21.15.7:0/74829551 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f882801eb00 con 0x7f8838063230 2023-10-07T07:57:35.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.231+0000 7f883f5e9700 1 -- 172.21.15.7:0/74829551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88380716c0 con 0x7f8838063230 2023-10-07T07:57:35.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.234+0000 7f88357fa700 1 -- 172.21.15.7:0/74829551 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f882802c020 con 0x7f8838063230 2023-10-07T07:57:35.238 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:35.236+0000 7f8a905001c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-07T07:57:35.250 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:35.249+0000 7f9216a671c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-07T07:57:35.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.307+0000 7f883f5e9700 1 -- 172.21.15.7:0/74829551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f883806ee60 con 0x7f8838063230 2023-10-07T07:57:35.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.308+0000 7f88357fa700 1 -- 172.21.15.7:0/74829551 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f882801ed90 con 0x7f8838063230 2023-10-07T07:57:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:35.309 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:35.016789+0000","last_up_change":"0.000000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:35.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.309+0000 7f883f5e9700 1 -- 172.21.15.7:0/74829551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838063230 msgr2=0x7f88381470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:35.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.309+0000 7f883f5e9700 1 --2- 172.21.15.7:0/74829551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838063230 0x7f88381470f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88280097c0 tx=0x7f8828019000 comp rx=0 tx=0).stop 2023-10-07T07:57:35.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.309+0000 7f883f5e9700 1 -- 172.21.15.7:0/74829551 shutdown_connections 2023-10-07T07:57:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.309+0000 7f883f5e9700 1 --2- 172.21.15.7:0/74829551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88380c9ec0 0x7f88380c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.309+0000 7f883f5e9700 1 --2- 172.21.15.7:0/74829551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838063230 0x7f88381470f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.309+0000 7f883f5e9700 1 --2- 172.21.15.7:0/74829551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88380c9af0 0x7f88380c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.309+0000 7f883f5e9700 1 -- 172.21.15.7:0/74829551 >> 172.21.15.7:0/74829551 conn(0x7f88380cd1a0 msgr2=0x7f8838155d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.309+0000 7f883f5e9700 1 -- 172.21.15.7:0/74829551 shutdown_connections 2023-10-07T07:57:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:35.309+0000 7f883f5e9700 1 -- 172.21.15.7:0/74829551 wait complete. 2023-10-07T07:57:35.323 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-07T07:57:35.323 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-07T07:57:35.392 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-10-07T07:57:35.391+0000 7fd52a3b1540 -1 Falling back to public interface 2023-10-07T07:57:35.401 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:35.400+0000 7f9216a671c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-07T07:57:35.419 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:35.418+0000 7f8a905001c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-07T07:57:35.478 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:35.477+0000 7f9216a671c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-07T07:57:35.497 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:35.496+0000 7f8a905001c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-07T07:57:35.573 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:35.572+0000 7f8a905001c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-07T07:57:35.653 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-10-07T07:57:35.652+0000 7fb697d50540 -1 Falling back to public interface 2023-10-07T07:57:35.654 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-10-07T07:57:35.653+0000 7f1c4755f540 -1 Falling back to public interface 2023-10-07T07:57:35.664 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-10-07T07:57:35.663+0000 7f807f8b9540 -1 Falling back to public interface 2023-10-07T07:57:35.664 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-10-07T07:57:35.663+0000 7fca486b7540 -1 Falling back to public interface 2023-10-07T07:57:35.664 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-10-07T07:57:35.663+0000 7f8079cc5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2023-10-07T07:57:35.671 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-10-07T07:57:35.669+0000 7fb88f627540 -1 Falling back to public interface 2023-10-07T07:57:35.673 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:35.672+0000 7f8a905001c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-07T07:57:35.677 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-10-07T07:57:35.676+0000 7f38018f1540 -1 Falling back to public interface 2023-10-07T07:57:35.682 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-10-07T07:57:35.679+0000 7fddf926c540 -1 Falling back to public interface 2023-10-07T07:57:35.684 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:35.683+0000 7f9216a671c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-07T07:57:35.771 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:35.770+0000 7f9216a671c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-07T07:57:35.852 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:35.851+0000 7f8a905001c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-07T07:57:35.856 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:35.855+0000 7f9216a671c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-07T07:57:35.932 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:35.930+0000 7f8a905001c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-07T07:57:35.967 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:35.966+0000 7f9216a671c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-07T07:57:36.128 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:36.169 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:36.167+0000 7f9216a671c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-07T07:57:36.220 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:36.218+0000 7f8a905001c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-07T07:57:36.252 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.251+0000 7fe940b02700 1 Processor -- start 2023-10-07T07:57:36.253 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.251+0000 7fe940b02700 1 -- start start 2023-10-07T07:57:36.253 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.252+0000 7fe940b02700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe93c159680 0x7fe93c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:36.254 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.252+0000 7fe940b02700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe93c06aa90 con 0x7fe93c0cab70 2023-10-07T07:57:36.254 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.252+0000 7fe940b02700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe93c0cb2e0 con 0x7fe93c1592b0 2023-10-07T07:57:36.254 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.252+0000 7fe940b02700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93c0cb5c0 con 0x7fe93c159680 2023-10-07T07:57:36.254 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe939d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe93c159680 0x7fe93c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:36.254 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe93ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe93c0cab70 0x7fe93c06af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43906/0 (socket says 172.21.15.7:43906) 2023-10-07T07:57:36.254 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe93ad9d700 1 -- 172.21.15.7:0/3413022856 learned_addr learned my addr 172.21.15.7:0/3413022856 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:36.255 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe93959a700 1 -- 172.21.15.7:0/3413022856 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726746344 0 0) 0x7fe93c0cb2e0 con 0x7fe93c1592b0 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe939d9b700 1 -- 172.21.15.7:0/3413022856 >> v1:172.21.15.172:6789/0 conn(0x7fe93c1592b0 legacy=0x7fe93c0a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe939d9b700 1 -- 172.21.15.7:0/3413022856 >> v1:172.21.15.7:6789/0 conn(0x7fe93c0cab70 legacy=0x7fe93c06af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe939d9b700 1 -- 172.21.15.7:0/3413022856 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c06a440 con 0x7fe93c159680 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe939d9b700 1 --2- 172.21.15.7:0/3413022856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c159680 0x7fe93c06eeb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe9300098e0 tx=0x7fe9300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48df8f0576fcd6a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe93959a700 1 -- 172.21.15.7:0/3413022856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930009e80 con 0x7fe93c159680 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe93959a700 1 -- 172.21.15.7:0/3413022856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93000b760 con 0x7fe93c159680 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.253+0000 7fe93959a700 1 -- 172.21.15.7:0/3413022856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93000bae0 con 0x7fe93c159680 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.254+0000 7fe940b02700 1 -- 172.21.15.7:0/3413022856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c159680 msgr2=0x7fe93c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.254+0000 7fe940b02700 1 --2- 172.21.15.7:0/3413022856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c159680 0x7fe93c06eeb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe9300098e0 tx=0x7fe9300091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.254+0000 7fe940b02700 1 -- 172.21.15.7:0/3413022856 shutdown_connections 2023-10-07T07:57:36.256 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.254+0000 7fe940b02700 1 --2- 172.21.15.7:0/3413022856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c159680 0x7fe93c06eeb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:36.257 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.254+0000 7fe940b02700 1 -- 172.21.15.7:0/3413022856 >> 172.21.15.7:0/3413022856 conn(0x7fe93c068100 msgr2=0x7fe93c068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:36.257 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.254+0000 7fe940b02700 1 -- 172.21.15.7:0/3413022856 shutdown_connections 2023-10-07T07:57:36.257 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.254+0000 7fe940b02700 1 -- 172.21.15.7:0/3413022856 wait complete. 2023-10-07T07:57:36.259 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.255+0000 7fe940b02700 1 Processor -- start 2023-10-07T07:57:36.259 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.255+0000 7fe940b02700 1 -- start start 2023-10-07T07:57:36.259 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe940b02700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe93c0cab70 0x7fe93c066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:36.259 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe940b02700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c1592b0 0x7fe93c067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:36.259 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe940b02700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe93c159680 0x7fe93c153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:36.259 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe940b02700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c15d6f0 con 0x7fe93c1592b0 2023-10-07T07:57:36.259 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe940b02700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c145f20 con 0x7fe93c159680 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe940b02700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c144ef0 con 0x7fe93c0cab70 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe939d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c1592b0 0x7fe93c067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe939d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c1592b0 0x7fe93c067530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40198/0 (socket says 172.21.15.7:40198) 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe939d9b700 1 -- 172.21.15.7:0/1073906171 learned_addr learned my addr 172.21.15.7:0/1073906171 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe93ad9d700 1 --2- 172.21.15.7:0/1073906171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe93c159680 0x7fe93c153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.256+0000 7fe93a59c700 1 --2- 172.21.15.7:0/1073906171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe93c0cab70 0x7fe93c066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe939d9b700 1 -- 172.21.15.7:0/1073906171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe93c0cab70 msgr2=0x7fe93c066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe939d9b700 1 --2- 172.21.15.7:0/1073906171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe93c0cab70 0x7fe93c066ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe939d9b700 1 -- 172.21.15.7:0/1073906171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe93c159680 msgr2=0x7fe93c153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe939d9b700 1 --2- 172.21.15.7:0/1073906171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe93c159680 0x7fe93c153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe939d9b700 1 -- 172.21.15.7:0/1073906171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe930009420 con 0x7fe93c1592b0 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe939d9b700 1 --2- 172.21.15.7:0/1073906171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c1592b0 0x7fe93c067530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe93c06dbe0 tx=0x7fe930016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:36.260 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe92a7fc700 1 -- 172.21.15.7:0/1073906171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93000bae0 con 0x7fe93c1592b0 2023-10-07T07:57:36.261 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe92a7fc700 1 -- 172.21.15.7:0/1073906171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93001c460 con 0x7fe93c1592b0 2023-10-07T07:57:36.261 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe92a7fc700 1 -- 172.21.15.7:0/1073906171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93001c800 con 0x7fe93c1592b0 2023-10-07T07:57:36.261 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe940b02700 1 -- 172.21.15.7:0/1073906171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93c1451d0 con 0x7fe93c1592b0 2023-10-07T07:57:36.261 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe940b02700 1 -- 172.21.15.7:0/1073906171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c145790 con 0x7fe93c1592b0 2023-10-07T07:57:36.261 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe92a7fc700 1 -- 172.21.15.7:0/1073906171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe93001a450 con 0x7fe93c1592b0 2023-10-07T07:57:36.261 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.257+0000 7fe92a7fc700 1 -- 172.21.15.7:0/1073906171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fe93001cdf0 con 0x7fe93c1592b0 2023-10-07T07:57:36.261 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.258+0000 7fe940b02700 1 -- 172.21.15.7:0/1073906171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93c06ce00 con 0x7fe93c1592b0 2023-10-07T07:57:36.262 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:36.260+0000 7f9216a671c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-07T07:57:36.262 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.260+0000 7fe92a7fc700 1 -- 172.21.15.7:0/1073906171 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe93002b060 con 0x7fe93c1592b0 2023-10-07T07:57:36.290 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:36.289+0000 7f8a905001c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-07T07:57:36.334 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.333+0000 7fe940b02700 1 -- 172.21.15.7:0/1073906171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe93c0716c0 con 0x7fe93c1592b0 2023-10-07T07:57:36.335 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.334+0000 7fe92a7fc700 1 -- 172.21.15.7:0/1073906171 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fe93001a890 con 0x7fe93c1592b0 2023-10-07T07:57:36.335 INFO:teuthology.misc.health.smithi007.stdout: 2023-10-07T07:57:36.335 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":10,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:35.016789+0000","last_up_change":"0.000000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:36.336 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.335+0000 7fe940b02700 1 -- 172.21.15.7:0/1073906171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c1592b0 msgr2=0x7fe93c067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:36.336 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.335+0000 7fe940b02700 1 --2- 172.21.15.7:0/1073906171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c1592b0 0x7fe93c067530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe93c06dbe0 tx=0x7fe930016000 comp rx=0 tx=0).stop 2023-10-07T07:57:36.337 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.335+0000 7fe940b02700 1 -- 172.21.15.7:0/1073906171 shutdown_connections 2023-10-07T07:57:36.337 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.335+0000 7fe940b02700 1 --2- 172.21.15.7:0/1073906171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe93c159680 0x7fe93c153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:36.337 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.335+0000 7fe940b02700 1 --2- 172.21.15.7:0/1073906171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe93c0cab70 0x7fe93c066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:36.337 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.335+0000 7fe940b02700 1 --2- 172.21.15.7:0/1073906171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe93c1592b0 0x7fe93c067530 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:36.337 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.335+0000 7fe940b02700 1 -- 172.21.15.7:0/1073906171 >> 172.21.15.7:0/1073906171 conn(0x7fe93c068100 msgr2=0x7fe93c0c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:36.337 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.335+0000 7fe940b02700 1 -- 172.21.15.7:0/1073906171 shutdown_connections 2023-10-07T07:57:36.337 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:36.335+0000 7fe940b02700 1 -- 172.21.15.7:0/1073906171 wait complete. 2023-10-07T07:57:36.348 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-07T07:57:36.588 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:36.587+0000 7f9216a671c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-07T07:57:36.668 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:36.667+0000 7f9216a671c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-07T07:57:36.797 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:36.795+0000 7f8a905001c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-07T07:57:37.112 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:37.111+0000 7f8a905001c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-07T07:57:37.236 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:37.235+0000 7f9216a671c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-07T07:57:37.252 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:37.251+0000 7f8a905001c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-07T07:57:37.594 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:37.593+0000 7f9216a671c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-07T07:57:37.609 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:37.608+0000 7f8a905001c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-07T07:57:37.751 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:37.749+0000 7f9216a671c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-07T07:57:37.751 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:37.750+0000 7f8a905001c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-07T07:57:37.905 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:37.903+0000 7f8a905001c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-07T07:57:38.004 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:38.003+0000 7f8a905001c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-07T07:57:38.167 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:38.166+0000 7f9216a671c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-07T07:57:38.328 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:38.327+0000 7f9216a671c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-07T07:57:38.586 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:38.585+0000 7f9216a671c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-07T07:57:38.698 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-10-07T07:57:38.697+0000 7f9216a671c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-07T07:57:38.904 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-10-07T07:57:38.903+0000 7fd52a3b1540 -1 osd.4 0 log_to_monitors true 2023-10-07T07:57:39.028 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:39.027+0000 7f8a54be1700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-07T07:57:39.031 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-10-07T07:57:39.030+0000 7f8a54be1700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-07T07:57:39.145 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-10-07T07:57:39.144+0000 7f1c4755f540 -1 osd.3 0 log_to_monitors true 2023-10-07T07:57:39.145 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-10-07T07:57:39.144+0000 7fb697d50540 -1 osd.5 0 log_to_monitors true 2023-10-07T07:57:39.191 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-10-07T07:57:39.190+0000 7f38018f1540 -1 osd.7 0 log_to_monitors true 2023-10-07T07:57:39.192 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-10-07T07:57:39.191+0000 7fca486b7540 -1 osd.1 0 log_to_monitors true 2023-10-07T07:57:39.193 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-10-07T07:57:39.191+0000 7fb88f627540 -1 osd.6 0 log_to_monitors true 2023-10-07T07:57:39.204 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-10-07T07:57:39.203+0000 7f807f8b9540 -1 osd.0 0 log_to_monitors true 2023-10-07T07:57:39.215 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-10-07T07:57:39.214+0000 7fddf926c540 -1 osd.2 0 log_to_monitors true 2023-10-07T07:57:41.624 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-10-07T07:57:41.622+0000 7f1c3f967700 -1 osd.3 0 waiting for initial osdmap 2023-10-07T07:57:41.633 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-10-07T07:57:41.630+0000 7fddefe71700 -1 osd.2 0 waiting for initial osdmap 2023-10-07T07:57:41.633 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-10-07T07:57:41.630+0000 7f1c36f5b700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T07:57:41.645 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-10-07T07:57:41.644+0000 7fca3f2bc700 -1 osd.1 0 waiting for initial osdmap 2023-10-07T07:57:41.649 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-10-07T07:57:41.648+0000 7fca380b3700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T07:57:41.652 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-10-07T07:57:41.652+0000 7f8077cc1700 -1 osd.0 0 waiting for initial osdmap 2023-10-07T07:57:41.653 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-10-07T07:57:41.652+0000 7fdde8c68700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T07:57:41.665 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-10-07T07:57:41.664+0000 7f806f2b5700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T07:57:42.227 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-10-07T07:57:42.224+0000 7fb88622c700 -1 osd.6 0 waiting for initial osdmap 2023-10-07T07:57:42.231 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-10-07T07:57:42.230+0000 7fb87f023700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T07:57:42.278 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-10-07T07:57:42.277+0000 7f37f84f6700 -1 osd.7 0 waiting for initial osdmap 2023-10-07T07:57:42.284 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-10-07T07:57:42.283+0000 7f37f12ed700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T07:57:42.288 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-10-07T07:57:42.287+0000 7fd520fb6700 -1 osd.4 0 waiting for initial osdmap 2023-10-07T07:57:42.292 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-10-07T07:57:42.291+0000 7fb68e955700 -1 osd.5 0 waiting for initial osdmap 2023-10-07T07:57:42.292 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-10-07T07:57:42.291+0000 7fd519dad700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T07:57:42.295 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-10-07T07:57:42.294+0000 7fb68774c700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T07:57:43.155 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:43.288 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.287+0000 7fa12a504700 1 Processor -- start 2023-10-07T07:57:43.288 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.287+0000 7fa12a504700 1 -- start start 2023-10-07T07:57:43.291 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa12a504700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1241592b0 0x7fa124159680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.291 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa12a504700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa12406eeb0 0x7fa1240cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.291 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa12a504700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa1240a9890 0x7fa1240a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.291 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa12a504700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1240cb5c0 con 0x7fa12406eeb0 2023-10-07T07:57:43.291 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa12a504700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12406f320 con 0x7fa1240a9890 2023-10-07T07:57:43.292 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa12a504700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1240cb0b0 con 0x7fa1241592b0 2023-10-07T07:57:43.292 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa1237fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa12406eeb0 0x7fa1240cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.292 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa1237fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa12406eeb0 0x7fa1240cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60056/0 (socket says 172.21.15.7:60056) 2023-10-07T07:57:43.292 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa1237fe700 1 -- 172.21.15.7:0/2190652032 learned_addr learned my addr 172.21.15.7:0/2190652032 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:43.292 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa128aa1700 1 --2- 172.21.15.7:0/2190652032 >> v2:172.21.15.172:3300/0 conn(0x7fa1240a9890 0x7fa1240a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.294 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa123fff700 1 --2- 172.21.15.7:0/2190652032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1241592b0 0x7fa124159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.294 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa1237fe700 1 -- 172.21.15.7:0/2190652032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1241592b0 msgr2=0x7fa124159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:43.294 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa1237fe700 1 --2- 172.21.15.7:0/2190652032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1241592b0 0x7fa124159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.294 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa1237fe700 1 -- 172.21.15.7:0/2190652032 >> v2:172.21.15.172:3300/0 conn(0x7fa1240a9890 msgr2=0x7fa1240a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:43.294 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa1237fe700 1 --2- 172.21.15.7:0/2190652032 >> v2:172.21.15.172:3300/0 conn(0x7fa1240a9890 0x7fa1240a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.288+0000 7fa1237fe700 1 -- 172.21.15.7:0/2190652032 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12406aa90 con 0x7fa12406eeb0 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa1237fe700 1 --2- 172.21.15.7:0/2190652032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa12406eeb0 0x7fa1240cab70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa1240680b0 tx=0x7fa10c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729d31fc9206664c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa122ffd700 1 -- 172.21.15.7:0/2190652032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c00b640 con 0x7fa12406eeb0 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa122ffd700 1 -- 172.21.15.7:0/2190652032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c00b7a0 con 0x7fa12406eeb0 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa122ffd700 1 -- 172.21.15.7:0/2190652032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c00bb60 con 0x7fa12406eeb0 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa12a504700 1 -- 172.21.15.7:0/2190652032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa12406eeb0 msgr2=0x7fa1240cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa12a504700 1 --2- 172.21.15.7:0/2190652032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa12406eeb0 0x7fa1240cab70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa1240680b0 tx=0x7fa10c009250 comp rx=0 tx=0).stop 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa12a504700 1 -- 172.21.15.7:0/2190652032 shutdown_connections 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa12a504700 1 --2- 172.21.15.7:0/2190652032 >> v2:172.21.15.172:3300/0 conn(0x7fa1240a9890 0x7fa1240a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa12a504700 1 --2- 172.21.15.7:0/2190652032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1241592b0 0x7fa124159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa12a504700 1 --2- 172.21.15.7:0/2190652032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa12406eeb0 0x7fa1240cab70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.295 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa12a504700 1 -- 172.21.15.7:0/2190652032 >> 172.21.15.7:0/2190652032 conn(0x7fa124068100 msgr2=0x7fa124068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:43.296 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa12a504700 1 -- 172.21.15.7:0/2190652032 shutdown_connections 2023-10-07T07:57:43.296 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.289+0000 7fa12a504700 1 -- 172.21.15.7:0/2190652032 wait complete. 2023-10-07T07:57:43.298 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.291+0000 7fa12a504700 1 Processor -- start 2023-10-07T07:57:43.298 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.291+0000 7fa12a504700 1 -- start start 2023-10-07T07:57:43.298 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa12a504700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa12406eeb0 0x7fa1241695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.298 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa12a504700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1240a9890 0x7fa12414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa12a504700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241592b0 0x7fa124150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa12a504700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12416a7d0 con 0x7fa1241592b0 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa12a504700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa124169bc0 con 0x7fa1240a9890 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa12a504700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa124169ea0 con 0x7fa12406eeb0 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa128aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241592b0 0x7fa124150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa128aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241592b0 0x7fa124150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60058/0 (socket says 172.21.15.7:60058) 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa128aa1700 1 -- 172.21.15.7:0/1916248099 learned_addr learned my addr 172.21.15.7:0/1916248099 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa1237fe700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1240a9890 0x7fa12414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa123fff700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa12406eeb0 0x7fa1241695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.299 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa128aa1700 1 -- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa12406eeb0 msgr2=0x7fa1241695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa128aa1700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa12406eeb0 0x7fa1241695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa128aa1700 1 -- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1240a9890 msgr2=0x7fa12414fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa128aa1700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1240a9890 0x7fa12414fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa128aa1700 1 -- 172.21.15.7:0/1916248099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0094f0 con 0x7fa1241592b0 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.292+0000 7fa128aa1700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241592b0 0x7fa124150700 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009e90 tx=0x7fa11c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.293+0000 7fa11bfff700 1 -- 172.21.15.7:0/1916248099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00cb70 con 0x7fa1241592b0 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.293+0000 7fa11bfff700 1 -- 172.21.15.7:0/1916248099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c00f040 con 0x7fa1241592b0 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.293+0000 7fa11bfff700 1 -- 172.21.15.7:0/1916248099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c014480 con 0x7fa1241592b0 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.293+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12416a180 con 0x7fa1241592b0 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.293+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1241501d0 con 0x7fa1241592b0 2023-10-07T07:57:43.300 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.294+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12406ce00 con 0x7fa1241592b0 2023-10-07T07:57:43.301 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.294+0000 7fa11bfff700 1 -- 172.21.15.7:0/1916248099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fa11c014830 con 0x7fa1241592b0 2023-10-07T07:57:43.301 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.295+0000 7fa11bfff700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa104066ea0 0x7fa104069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.301 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.295+0000 7fa11bfff700 1 -- 172.21.15.7:0/1916248099 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fa11c084910 con 0x7fa1241592b0 2023-10-07T07:57:43.301 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.295+0000 7fa123fff700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa104066ea0 0x7fa104069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.301 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.296+0000 7fa123fff700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa104066ea0 0x7fa104069350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa124158550 tx=0x7fa1140091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:43.301 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.300+0000 7fa11bfff700 1 -- 172.21.15.7:0/1916248099 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa11c052130 con 0x7fa1241592b0 2023-10-07T07:57:43.506 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.505+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1240716c0 con 0x7fa1241592b0 2023-10-07T07:57:43.507 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.506+0000 7fa11bfff700 1 -- 172.21.15.7:0/1916248099 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10444 (secure 0 0 0) 0x7fa11c055750 con 0x7fa1241592b0 2023-10-07T07:57:43.507 INFO:teuthology.misc.health.smithi007.stdout: 2023-10-07T07:57:43.507 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":13,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:42.649003+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:43.510 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.508+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa104066ea0 msgr2=0x7fa104069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.510 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.508+0000 7fa12a504700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa104066ea0 0x7fa104069350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa124158550 tx=0x7fa1140091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.510 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.508+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241592b0 msgr2=0x7fa124150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.510 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.508+0000 7fa12a504700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241592b0 0x7fa124150700 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009e90 tx=0x7fa11c00b040 comp rx=0 tx=0).stop 2023-10-07T07:57:43.510 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.508+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 shutdown_connections 2023-10-07T07:57:43.510 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.508+0000 7fa12a504700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa104066ea0 0x7fa104069350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.510 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.509+0000 7fa12a504700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241592b0 0x7fa124150700 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.511 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.509+0000 7fa12a504700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa12406eeb0 0x7fa1241695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.511 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.509+0000 7fa12a504700 1 --2- 172.21.15.7:0/1916248099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1240a9890 0x7fa12414fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.511 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.509+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 >> 172.21.15.7:0/1916248099 conn(0x7fa124068100 msgr2=0x7fa1240afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:43.511 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.509+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 shutdown_connections 2023-10-07T07:57:43.511 INFO:teuthology.misc.health.smithi007.stderr:2023-10-07T07:57:43.509+0000 7fa12a504700 1 -- 172.21.15.7:0/1916248099 wait complete. 2023-10-07T07:57:43.522 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-07T07:57:43.522 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-07T07:57:43.522 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-07T07:57:43.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-07T07:57:43.525 INFO:tasks.ceph.mds.a:Started 2023-10-07T07:57:43.525 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-07T07:57:43.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-07T07:57:43.527 INFO:tasks.ceph.mds.c:Started 2023-10-07T07:57:43.527 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-07T07:57:43.527 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-07T07:57:43.529 INFO:tasks.ceph.mds.b:Started 2023-10-07T07:57:43.530 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-07T07:57:43.530 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-07T07:57:43.532 INFO:tasks.ceph.mds.d:Started 2023-10-07T07:57:43.532 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-07T07:57:43.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:43.610 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2023-10-07T07:57:43.612 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2023-10-07T07:57:43.615 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2023-10-07T07:57:43.615 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2023-10-07T07:57:43.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.669+0000 7f4c975c6700 1 Processor -- start 2023-10-07T07:57:43.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.670+0000 7f4c975c6700 1 -- start start 2023-10-07T07:57:43.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c975c6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c90157470 0x7f4c9006b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c975c6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4c9006bb60 0x7f4c900c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c975c6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c900c8f10 con 0x7f4c901570a0 2023-10-07T07:57:43.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c975c6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9006bfe0 con 0x7f4c90157470 2023-10-07T07:57:43.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c975c6700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c900c3b80 con 0x7f4c9006bb60 2023-10-07T07:57:43.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c94b61700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c90157470 0x7f4c9006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c95362700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4c901570a0 0x7f4c900c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38604/0 (socket says 172.21.15.7:38604) 2023-10-07T07:57:43.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c95362700 1 -- 172.21.15.7:0/2449647721 learned_addr learned my addr 172.21.15.7:0/2449647721 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:43.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c95b63700 1 --2- 172.21.15.7:0/2449647721 >> v2:172.21.15.172:3300/0 conn(0x7f4c9006bb60 0x7f4c900c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c94b61700 1 -- 172.21.15.7:0/2449647721 >> v2:172.21.15.172:3300/0 conn(0x7f4c9006bb60 msgr2=0x7f4c900c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:43.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c94b61700 1 --2- 172.21.15.7:0/2449647721 >> v2:172.21.15.172:3300/0 conn(0x7f4c9006bb60 0x7f4c900c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c94b61700 1 -- 172.21.15.7:0/2449647721 >> v1:172.21.15.7:6789/0 conn(0x7f4c901570a0 legacy=0x7f4c900c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c94b61700 1 -- 172.21.15.7:0/2449647721 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900c2f70 con 0x7f4c90157470 2023-10-07T07:57:43.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.671+0000 7f4c95b63700 1 --2- 172.21.15.7:0/2449647721 >> v2:172.21.15.172:3300/0 conn(0x7f4c9006bb60 0x7f4c900c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:57:43.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c94b61700 1 --2- 172.21.15.7:0/2449647721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c90157470 0x7f4c9006b620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c900cf190 tx=0x7f4c800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be04ba8d58db819 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:43.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c87fff700 1 -- 172.21.15.7:0/2449647721 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000b6b0 con 0x7f4c90157470 2023-10-07T07:57:43.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c87fff700 1 -- 172.21.15.7:0/2449647721 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8000b810 con 0x7f4c90157470 2023-10-07T07:57:43.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c87fff700 1 -- 172.21.15.7:0/2449647721 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000bbb0 con 0x7f4c90157470 2023-10-07T07:57:43.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c975c6700 1 -- 172.21.15.7:0/2449647721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c90157470 msgr2=0x7f4c9006b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c975c6700 1 --2- 172.21.15.7:0/2449647721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c90157470 0x7f4c9006b620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c900cf190 tx=0x7f4c800091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c975c6700 1 -- 172.21.15.7:0/2449647721 shutdown_connections 2023-10-07T07:57:43.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c975c6700 1 --2- 172.21.15.7:0/2449647721 >> v2:172.21.15.172:3300/0 conn(0x7f4c9006bb60 0x7f4c900c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c975c6700 1 --2- 172.21.15.7:0/2449647721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c90157470 0x7f4c9006b620 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.672+0000 7f4c975c6700 1 -- 172.21.15.7:0/2449647721 >> 172.21.15.7:0/2449647721 conn(0x7f4c90067ac0 msgr2=0x7f4c900ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:43.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.673+0000 7f4c975c6700 1 -- 172.21.15.7:0/2449647721 shutdown_connections 2023-10-07T07:57:43.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.673+0000 7f4c975c6700 1 -- 172.21.15.7:0/2449647721 wait complete. 2023-10-07T07:57:43.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.674+0000 7f4c975c6700 1 Processor -- start 2023-10-07T07:57:43.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.674+0000 7f4c975c6700 1 -- start start 2023-10-07T07:57:43.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.675+0000 7f4c975c6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c9006bb60 0x7f4c900c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.675+0000 7f4c975c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901570a0 0x7f4c900b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.675+0000 7f4c975c6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c90157470 0x7f4c90167790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.675+0000 7f4c975c6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900c5b60 con 0x7f4c901570a0 2023-10-07T07:57:43.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.675+0000 7f4c975c6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900c6980 con 0x7f4c9006bb60 2023-10-07T07:57:43.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.675+0000 7f4c975c6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900c5d70 con 0x7f4c90157470 2023-10-07T07:57:43.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c94b61700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901570a0 0x7f4c900b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c94b61700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901570a0 0x7f4c900b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60116/0 (socket says 172.21.15.7:60116) 2023-10-07T07:57:43.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c94b61700 1 -- 172.21.15.7:0/4250564259 learned_addr learned my addr 172.21.15.7:0/4250564259 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:43.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c95362700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c9006bb60 0x7f4c900c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c95b63700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c90157470 0x7f4c90167790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c94b61700 1 -- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c90157470 msgr2=0x7f4c90167790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c94b61700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c90157470 0x7f4c90167790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c94b61700 1 -- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c9006bb60 msgr2=0x7f4c900c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c94b61700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c9006bb60 0x7f4c900c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c94b61700 1 -- 172.21.15.7:0/4250564259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80009420 con 0x7f4c901570a0 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c94b61700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901570a0 0x7f4c900b3610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c9013a860 tx=0x7f4c80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c857fa700 1 -- 172.21.15.7:0/4250564259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80011420 con 0x7f4c901570a0 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c857fa700 1 -- 172.21.15.7:0/4250564259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80011580 con 0x7f4c901570a0 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c857fa700 1 -- 172.21.15.7:0/4250564259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80011920 con 0x7f4c901570a0 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c900c6050 con 0x7f4c901570a0 2023-10-07T07:57:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.676+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c900c6610 con 0x7f4c901570a0 2023-10-07T07:57:43.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.677+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c900c6ac0 con 0x7f4c901570a0 2023-10-07T07:57:43.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.678+0000 7f4c857fa700 1 -- 172.21.15.7:0/4250564259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f4c80011cd0 con 0x7f4c901570a0 2023-10-07T07:57:43.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.678+0000 7f4c857fa700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c74066ea0 0x7f4c74069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:43.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.678+0000 7f4c857fa700 1 -- 172.21.15.7:0/4250564259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f4c80085950 con 0x7f4c901570a0 2023-10-07T07:57:43.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.678+0000 7f4c95362700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c74066ea0 0x7f4c74069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:43.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.679+0000 7f4c95362700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c74066ea0 0x7f4c74069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4c90156540 tx=0x7f4c8c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:43.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.683+0000 7f4c857fa700 1 -- 172.21.15.7:0/4250564259 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c900c6ac0 con 0x7f4c901570a0 2023-10-07T07:57:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.887+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c900c6ac0 con 0x7f4c901570a0 2023-10-07T07:57:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.888+0000 7f4c857fa700 1 -- 172.21.15.7:0/4250564259 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10628 (secure 0 0 0) 0x7f4c900c6ac0 con 0x7f4c901570a0 2023-10-07T07:57:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:43.890 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":14,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:43.655320+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.890+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c74066ea0 msgr2=0x7f4c74069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.890+0000 7f4c975c6700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c74066ea0 0x7f4c74069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4c90156540 tx=0x7f4c8c009040 comp rx=0 tx=0).stop 2023-10-07T07:57:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.890+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901570a0 msgr2=0x7f4c900b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.890+0000 7f4c975c6700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901570a0 0x7f4c900b3610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c9013a860 tx=0x7f4c80017000 comp rx=0 tx=0).stop 2023-10-07T07:57:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.891+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 shutdown_connections 2023-10-07T07:57:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.891+0000 7f4c975c6700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c74066ea0 0x7f4c74069350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.891+0000 7f4c975c6700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c90157470 0x7f4c90167790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.891+0000 7f4c975c6700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c9006bb60 0x7f4c900c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.891+0000 7f4c975c6700 1 --2- 172.21.15.7:0/4250564259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901570a0 0x7f4c900b3610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.891+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 >> 172.21.15.7:0/4250564259 conn(0x7f4c90067ac0 msgr2=0x7f4c90139ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.891+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 shutdown_connections 2023-10-07T07:57:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:43.891+0000 7f4c975c6700 1 -- 172.21.15.7:0/4250564259 wait complete. 2023-10-07T07:57:43.906 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-10-07T07:57:43.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:57:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.037+0000 7f596ee93700 1 Processor -- start 2023-10-07T07:57:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.037+0000 7f596ee93700 1 -- start start 2023-10-07T07:57:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596ee93700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5968155f20 0x7f59680c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596ee93700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59681562f0 0x7f596806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596ee93700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f596806e2c0 0x7f596806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596ee93700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596806ec60 con 0x7f596806e2c0 2023-10-07T07:57:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596ee93700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596806b700 con 0x7f59681562f0 2023-10-07T07:57:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596ee93700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596806aaf0 con 0x7f5968155f20 2023-10-07T07:57:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596d430700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f596806e2c0 0x7f596806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596d430700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f596806e2c0 0x7f596806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60126/0 (socket says 172.21.15.7:60126) 2023-10-07T07:57:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596d430700 1 -- 172.21.15.7:0/177023639 learned_addr learned my addr 172.21.15.7:0/177023639 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f5967fff700 1 --2- 172.21.15.7:0/177023639 >> v2:172.21.15.172:3300/0 conn(0x7f59681562f0 0x7f596806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596d430700 1 -- 172.21.15.7:0/177023639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5968155f20 msgr2=0x7f59680c92b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596d430700 1 --2- 172.21.15.7:0/177023639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5968155f20 0x7f59680c92b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596d430700 1 -- 172.21.15.7:0/177023639 >> v2:172.21.15.172:3300/0 conn(0x7f59681562f0 msgr2=0x7f596806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596d430700 1 --2- 172.21.15.7:0/177023639 >> v2:172.21.15.172:3300/0 conn(0x7f59681562f0 0x7f596806a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596d430700 1 -- 172.21.15.7:0/177023639 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596806add0 con 0x7f596806e2c0 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.038+0000 7f596d430700 1 --2- 172.21.15.7:0/177023639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806e2c0 0x7f596806e720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f59680caac0 tx=0x7f596000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359a712d068f784a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f59677fe700 1 -- 172.21.15.7:0/177023639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960010070 con 0x7f596806e2c0 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f59677fe700 1 -- 172.21.15.7:0/177023639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596000de90 con 0x7f596806e2c0 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f59677fe700 1 -- 172.21.15.7:0/177023639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596000b320 con 0x7f596806e2c0 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f596ee93700 1 -- 172.21.15.7:0/177023639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806e2c0 msgr2=0x7f596806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f596ee93700 1 --2- 172.21.15.7:0/177023639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806e2c0 0x7f596806e720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f59680caac0 tx=0x7f596000ba50 comp rx=0 tx=0).stop 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f596ee93700 1 -- 172.21.15.7:0/177023639 shutdown_connections 2023-10-07T07:57:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f596ee93700 1 --2- 172.21.15.7:0/177023639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596806e2c0 0x7f596806e720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f596ee93700 1 --2- 172.21.15.7:0/177023639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5968155f20 0x7f59680c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f596ee93700 1 --2- 172.21.15.7:0/177023639 >> v2:172.21.15.172:3300/0 conn(0x7f59681562f0 0x7f596806a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f596ee93700 1 -- 172.21.15.7:0/177023639 >> 172.21.15.7:0/177023639 conn(0x7f5968068bd0 msgr2=0x7f59680ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f596ee93700 1 -- 172.21.15.7:0/177023639 shutdown_connections 2023-10-07T07:57:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.039+0000 7f596ee93700 1 -- 172.21.15.7:0/177023639 wait complete. 2023-10-07T07:57:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.041+0000 7f596ee93700 1 Processor -- start 2023-10-07T07:57:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.041+0000 7f596ee93700 1 -- start start 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f596ee93700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596806e2c0 0x7f5968152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f596ee93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968155f20 0x7f59680c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f596ee93700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59681562f0 0x7f5968164d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f596ee93700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59680c63e0 con 0x7f5968155f20 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f596ee93700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5968156810 con 0x7f59681562f0 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f596ee93700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5968153660 con 0x7f596806e2c0 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5967fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968155f20 0x7f59680c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5967fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968155f20 0x7f59680c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60142/0 (socket says 172.21.15.7:60142) 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5967fff700 1 -- 172.21.15.7:0/3623235662 learned_addr learned my addr 172.21.15.7:0/3623235662 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f596d430700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59681562f0 0x7f5968164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f596cc2f700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596806e2c0 0x7f5968152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5967fff700 1 -- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596806e2c0 msgr2=0x7f5968152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5967fff700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596806e2c0 0x7f5968152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5967fff700 1 -- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59681562f0 msgr2=0x7f5968164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5967fff700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59681562f0 0x7f5968164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5967fff700 1 -- 172.21.15.7:0/3623235662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596000bcf0 con 0x7f5968155f20 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5967fff700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968155f20 0x7f59680c5ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f595400d860 tx=0x7f5954008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5964ff9700 1 -- 172.21.15.7:0/3623235662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540125f0 con 0x7f5968155f20 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5964ff9700 1 -- 172.21.15.7:0/3623235662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5954012750 con 0x7f5968155f20 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5968152a50 con 0x7f5968155f20 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.042+0000 7f5964ff9700 1 -- 172.21.15.7:0/3623235662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954012b10 con 0x7f5968155f20 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.043+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5968153010 con 0x7f5968155f20 2023-10-07T07:57:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.043+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59680716c0 con 0x7f5968155f20 2023-10-07T07:57:44.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.044+0000 7f5964ff9700 1 -- 172.21.15.7:0/3623235662 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f595401e460 con 0x7f5968155f20 2023-10-07T07:57:44.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.044+0000 7f5964ff9700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f594c066ea0 0x7f594c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:44.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.044+0000 7f5964ff9700 1 -- 172.21.15.7:0/3623235662 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f5954084470 con 0x7f5968155f20 2023-10-07T07:57:44.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.045+0000 7f596cc2f700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f594c066ea0 0x7f594c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:44.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.045+0000 7f596cc2f700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f594c066ea0 0x7f594c069350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f595c006fd0 tx=0x7f595c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:44.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.049+0000 7f5964ff9700 1 -- 172.21.15.7:0/3623235662 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5954051c90 con 0x7f5968155f20 2023-10-07T07:57:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.300+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f596806f1f0 con 0x7f5968155f20 2023-10-07T07:57:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.301+0000 7f5964ff9700 1 -- 172.21.15.7:0/3623235662 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1253 (secure 0 0 0) 0x7f59540552b0 con 0x7f5968155f20 2023-10-07T07:57:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:44.303 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-07T07:57:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f594c066ea0 msgr2=0x7f594c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f594c066ea0 0x7f594c069350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f595c006fd0 tx=0x7f595c0090f0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968155f20 msgr2=0x7f59680c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968155f20 0x7f59680c5ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f595400d860 tx=0x7f5954008040 comp rx=0 tx=0).stop 2023-10-07T07:57:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 shutdown_connections 2023-10-07T07:57:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f594c066ea0 0x7f594c069350 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59681562f0 0x7f5968164d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f596806e2c0 0x7f5968152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 --2- 172.21.15.7:0/3623235662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5968155f20 0x7f59680c5ea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 >> 172.21.15.7:0/3623235662 conn(0x7f5968068bd0 msgr2=0x7f59680ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 shutdown_connections 2023-10-07T07:57:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:44.303+0000 7f596ee93700 1 -- 172.21.15.7:0/3623235662 wait complete. 2023-10-07T07:57:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2023-10-07T07:57:46.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:57:46.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.452+0000 7fd14d36d700 1 Processor -- start 2023-10-07T07:57:46.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.452+0000 7fd14d36d700 1 -- start start 2023-10-07T07:57:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd14d36d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd148156050 0x7fd1480aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd14d36d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1480c8cd0 0x7fd1480c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd14d36d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 0x7fd148159d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd14d36d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1480ca240 con 0x7fd1480c8cd0 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd14d36d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1480af590 con 0x7fd148156050 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd14d36d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480b03f0 con 0x7fd1480c9de0 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd1467fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1480c8cd0 0x7fd1480c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd1467fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1480c8cd0 0x7fd1480c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60146/0 (socket says 172.21.15.7:60146) 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd1467fc700 1 -- 172.21.15.7:0/1396082542 learned_addr learned my addr 172.21.15.7:0/1396082542 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd1477fe700 1 --2- 172.21.15.7:0/1396082542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 0x7fd148159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.454+0000 7fd146ffd700 1 --2- 172.21.15.7:0/1396082542 >> v2:172.21.15.172:3300/0 conn(0x7fd148156050 0x7fd1480aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.455+0000 7fd1467fc700 1 -- 172.21.15.7:0/1396082542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 msgr2=0x7fd148159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.455+0000 7fd1467fc700 1 --2- 172.21.15.7:0/1396082542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 0x7fd148159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.455+0000 7fd1467fc700 1 -- 172.21.15.7:0/1396082542 >> v2:172.21.15.172:3300/0 conn(0x7fd148156050 msgr2=0x7fd1480aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.455+0000 7fd1467fc700 1 --2- 172.21.15.7:0/1396082542 >> v2:172.21.15.172:3300/0 conn(0x7fd148156050 0x7fd1480aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.455+0000 7fd1467fc700 1 -- 172.21.15.7:0/1396082542 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480af7e0 con 0x7fd1480c8cd0 2023-10-07T07:57:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.455+0000 7fd1467fc700 1 --2- 172.21.15.7:0/1396082542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 0x7fd1480c98a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd148063fa0 tx=0x7fd13c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18de574ef71f623 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.455+0000 7fd145ffb700 1 -- 172.21.15.7:0/1396082542 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00b860 con 0x7fd1480c8cd0 2023-10-07T07:57:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.455+0000 7fd145ffb700 1 -- 172.21.15.7:0/1396082542 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c00b9c0 con 0x7fd1480c8cd0 2023-10-07T07:57:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.455+0000 7fd145ffb700 1 -- 172.21.15.7:0/1396082542 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00bd60 con 0x7fd1480c8cd0 2023-10-07T07:57:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.456+0000 7fd14d36d700 1 -- 172.21.15.7:0/1396082542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 msgr2=0x7fd1480c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.456+0000 7fd14d36d700 1 --2- 172.21.15.7:0/1396082542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 0x7fd1480c98a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd148063fa0 tx=0x7fd13c009250 comp rx=0 tx=0).stop 2023-10-07T07:57:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.456+0000 7fd14d36d700 1 -- 172.21.15.7:0/1396082542 shutdown_connections 2023-10-07T07:57:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.456+0000 7fd14d36d700 1 --2- 172.21.15.7:0/1396082542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 0x7fd148159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.456+0000 7fd14d36d700 1 --2- 172.21.15.7:0/1396082542 >> v2:172.21.15.172:3300/0 conn(0x7fd148156050 0x7fd1480aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.456+0000 7fd14d36d700 1 --2- 172.21.15.7:0/1396082542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 0x7fd1480c98a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.456+0000 7fd14d36d700 1 -- 172.21.15.7:0/1396082542 >> 172.21.15.7:0/1396082542 conn(0x7fd1480cd1a0 msgr2=0x7fd1480c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.456+0000 7fd14d36d700 1 -- 172.21.15.7:0/1396082542 shutdown_connections 2023-10-07T07:57:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.456+0000 7fd14d36d700 1 -- 172.21.15.7:0/1396082542 wait complete. 2023-10-07T07:57:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.458+0000 7fd14d36d700 1 Processor -- start 2023-10-07T07:57:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.458+0000 7fd14d36d700 1 -- start start 2023-10-07T07:57:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd14d36d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 0x7fd1480c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd14d36d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 0x7fd1480c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd14d36d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd148156050 0x7fd14814f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd14d36d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480c1b60 con 0x7fd1480c8cd0 2023-10-07T07:57:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd14d36d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480c35f0 con 0x7fd148156050 2023-10-07T07:57:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd14d36d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480c38d0 con 0x7fd1480c9de0 2023-10-07T07:57:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd146ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 0x7fd1480c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd146ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 0x7fd1480c2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60152/0 (socket says 172.21.15.7:60152) 2023-10-07T07:57:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd146ffd700 1 -- 172.21.15.7:0/3633445793 learned_addr learned my addr 172.21.15.7:0/3633445793 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd1467fc700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 0x7fd1480c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd1477fe700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd148156050 0x7fd14814f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd146ffd700 1 -- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 msgr2=0x7fd1480c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd146ffd700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 0x7fd1480c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd146ffd700 1 -- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd148156050 msgr2=0x7fd14814f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd146ffd700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd148156050 0x7fd14814f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd146ffd700 1 -- 172.21.15.7:0/3633445793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c0094f0 con 0x7fd1480c8cd0 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd146ffd700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 0x7fd1480c2dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd13800d890 tx=0x7fd138008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.459+0000 7fd12effd700 1 -- 172.21.15.7:0/3633445793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1380125f0 con 0x7fd1480c8cd0 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.460+0000 7fd12effd700 1 -- 172.21.15.7:0/3633445793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd138012750 con 0x7fd1480c8cd0 2023-10-07T07:57:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.460+0000 7fd12effd700 1 -- 172.21.15.7:0/3633445793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138012af0 con 0x7fd1480c8cd0 2023-10-07T07:57:46.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.460+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1480c1510 con 0x7fd1480c8cd0 2023-10-07T07:57:46.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.460+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1480c3300 con 0x7fd1480c8cd0 2023-10-07T07:57:46.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.461+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1480716c0 con 0x7fd1480c8cd0 2023-10-07T07:57:46.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.461+0000 7fd12effd700 1 -- 172.21.15.7:0/3633445793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fd13801e460 con 0x7fd1480c8cd0 2023-10-07T07:57:46.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.462+0000 7fd12effd700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd134066ea0 0x7fd134069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.462+0000 7fd12effd700 1 -- 172.21.15.7:0/3633445793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd138084650 con 0x7fd1480c8cd0 2023-10-07T07:57:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.462+0000 7fd1467fc700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd134066ea0 0x7fd134069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.462+0000 7fd1467fc700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd134066ea0 0x7fd134069350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd148063fa0 tx=0x7fd13c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.467+0000 7fd12effd700 1 -- 172.21.15.7:0/3633445793 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd138051c50 con 0x7fd1480c8cd0 2023-10-07T07:57:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.718+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd14806ef60 con 0x7fd1480c8cd0 2023-10-07T07:57:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.719+0000 7fd12effd700 1 -- 172.21.15.7:0/3633445793 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7fd138055270 con 0x7fd1480c8cd0 2023-10-07T07:57:46.722 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:46.722 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2318752844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2318752844},{"type":"v1","addr":"172.21.15.172:6836","nonce":2318752844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1231430544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1231430544},{"type":"v1","addr":"172.21.15.172:6837","nonce":1231430544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1421504798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1421504798},{"type":"v1","addr":"172.21.15.7:6833","nonce":1421504798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-07T07:57:46.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd134066ea0 msgr2=0x7fd134069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:46.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd134066ea0 0x7fd134069350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd148063fa0 tx=0x7fd13c017000 comp rx=0 tx=0).stop 2023-10-07T07:57:46.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 msgr2=0x7fd1480c2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 0x7fd1480c2dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd13800d890 tx=0x7fd138008040 comp rx=0 tx=0).stop 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 shutdown_connections 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd134066ea0 0x7fd134069350 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd148156050 0x7fd14814f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1480c8cd0 0x7fd1480c2dc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 --2- 172.21.15.7:0/3633445793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1480c9de0 0x7fd1480c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.721+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 >> 172.21.15.7:0/3633445793 conn(0x7fd1480cd1a0 msgr2=0x7fd1480ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.722+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 shutdown_connections 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.722+0000 7fd14d36d700 1 -- 172.21.15.7:0/3633445793 wait complete. 2023-10-07T07:57:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2023-10-07T07:57:46.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.868+0000 7fe242480700 1 Processor -- start 2023-10-07T07:57:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.868+0000 7fe242480700 1 -- start start 2023-10-07T07:57:46.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.869+0000 7fe242480700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe23c0ced30 0x7fe23c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.869+0000 7fe242480700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe23c1596e0 0x7fe23c06b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.869+0000 7fe242480700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23c068100 con 0x7fe23c159310 2023-10-07T07:57:46.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.869+0000 7fe242480700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c068350 con 0x7fe23c0ced30 2023-10-07T07:57:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.869+0000 7fe242480700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c068630 con 0x7fe23c1596e0 2023-10-07T07:57:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.869+0000 7fe23bfff700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe23c159310 0x7fe23c0c3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:38746/0 (socket says 172.21.15.7:38746) 2023-10-07T07:57:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.869+0000 7fe23bfff700 1 -- 172.21.15.7:0/1581568481 learned_addr learned my addr 172.21.15.7:0/1581568481 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.869+0000 7fe23b7fe700 1 --2- 172.21.15.7:0/1581568481 >> v2:172.21.15.7:3300/0 conn(0x7fe23c0ced30 0x7fe23c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.869+0000 7fe240a1d700 1 --2- 172.21.15.7:0/1581568481 >> v2:172.21.15.172:3300/0 conn(0x7fe23c1596e0 0x7fe23c06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe23affd700 1 -- 172.21.15.7:0/1581568481 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932025730 0 0) 0x7fe23c068100 con 0x7fe23c159310 2023-10-07T07:57:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe23affd700 1 -- 172.21.15.7:0/1581568481 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe224003570 con 0x7fe23c159310 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe240a1d700 1 -- 172.21.15.7:0/1581568481 >> v2:172.21.15.7:3300/0 conn(0x7fe23c0ced30 msgr2=0x7fe23c0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe240a1d700 1 --2- 172.21.15.7:0/1581568481 >> v2:172.21.15.7:3300/0 conn(0x7fe23c0ced30 0x7fe23c0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe240a1d700 1 -- 172.21.15.7:0/1581568481 >> v1:172.21.15.172:6789/0 conn(0x7fe23c159310 legacy=0x7fe23c0c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe240a1d700 1 -- 172.21.15.7:0/1581568481 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c068910 con 0x7fe23c1596e0 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe240a1d700 1 --2- 172.21.15.7:0/1581568481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe23c1596e0 0x7fe23c06b620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0c85e0 tx=0x7fe234009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dae178b8a618db2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe23affd700 1 -- 172.21.15.7:0/1581568481 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400e070 con 0x7fe23c1596e0 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe23affd700 1 -- 172.21.15.7:0/1581568481 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23400bab0 con 0x7fe23c1596e0 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.870+0000 7fe23affd700 1 -- 172.21.15.7:0/1581568481 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400be70 con 0x7fe23c1596e0 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.871+0000 7fe242480700 1 -- 172.21.15.7:0/1581568481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe23c1596e0 msgr2=0x7fe23c06b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.871+0000 7fe242480700 1 --2- 172.21.15.7:0/1581568481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe23c1596e0 0x7fe23c06b620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0c85e0 tx=0x7fe234009780 comp rx=0 tx=0).stop 2023-10-07T07:57:46.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.871+0000 7fe242480700 1 -- 172.21.15.7:0/1581568481 shutdown_connections 2023-10-07T07:57:46.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.871+0000 7fe242480700 1 --2- 172.21.15.7:0/1581568481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe23c1596e0 0x7fe23c06b620 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.871+0000 7fe242480700 1 --2- 172.21.15.7:0/1581568481 >> v2:172.21.15.7:3300/0 conn(0x7fe23c0ced30 0x7fe23c0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.871+0000 7fe242480700 1 -- 172.21.15.7:0/1581568481 >> 172.21.15.7:0/1581568481 conn(0x7fe23c067ac0 msgr2=0x7fe23c0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:46.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.871+0000 7fe242480700 1 -- 172.21.15.7:0/1581568481 shutdown_connections 2023-10-07T07:57:46.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.871+0000 7fe242480700 1 -- 172.21.15.7:0/1581568481 wait complete. 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.872+0000 7fe242480700 1 Processor -- start 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.873+0000 7fe242480700 1 -- start start 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.873+0000 7fe242480700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe23c159310 0x7fe23c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.873+0000 7fe242480700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe23c1596e0 0x7fe23c13b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.873+0000 7fe242480700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c13b4c0 0x7fe23c0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.873+0000 7fe242480700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c1526a0 con 0x7fe23c13b4c0 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.873+0000 7fe242480700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c15a5f0 con 0x7fe23c159310 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.873+0000 7fe242480700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c153660 con 0x7fe23c1596e0 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.873+0000 7fe240a1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c13b4c0 0x7fe23c0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe240a1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c13b4c0 0x7fe23c0b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60174/0 (socket says 172.21.15.7:60174) 2023-10-07T07:57:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe240a1d700 1 -- 172.21.15.7:0/719399240 learned_addr learned my addr 172.21.15.7:0/719399240 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe23bfff700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe23c159310 0x7fe23c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe23b7fe700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe23c1596e0 0x7fe23c13b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe240a1d700 1 -- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe23c1596e0 msgr2=0x7fe23c13b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe240a1d700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe23c1596e0 0x7fe23c13b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe240a1d700 1 -- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe23c159310 msgr2=0x7fe23c15a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe240a1d700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe23c159310 0x7fe23c15a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe240a1d700 1 -- 172.21.15.7:0/719399240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe234009a20 con 0x7fe23c13b4c0 2023-10-07T07:57:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.874+0000 7fe240a1d700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c13b4c0 0x7fe23c0b0e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe23c068770 tx=0x7fe234019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:46.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.938+0000 7fe223fff700 1 -- 172.21.15.7:0/719399240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234012dd0 con 0x7fe23c13b4c0 2023-10-07T07:57:46.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.938+0000 7fe223fff700 1 -- 172.21.15.7:0/719399240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234027470 con 0x7fe23c13b4c0 2023-10-07T07:57:46.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.938+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c1528a0 con 0x7fe23c13b4c0 2023-10-07T07:57:46.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.938+0000 7fe223fff700 1 -- 172.21.15.7:0/719399240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234027830 con 0x7fe23c13b4c0 2023-10-07T07:57:46.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.938+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c152e60 con 0x7fe23c13b4c0 2023-10-07T07:57:46.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.939+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c0716c0 con 0x7fe23c13b4c0 2023-10-07T07:57:46.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.940+0000 7fe223fff700 1 -- 172.21.15.7:0/719399240 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fe234024070 con 0x7fe23c13b4c0 2023-10-07T07:57:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.940+0000 7fe223fff700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe228066ea0 0x7fe228069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.940+0000 7fe223fff700 1 -- 172.21.15.7:0/719399240 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe234086d70 con 0x7fe23c13b4c0 2023-10-07T07:57:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.940+0000 7fe23bfff700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe228066ea0 0x7fe228069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:46.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.941+0000 7fe23bfff700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe228066ea0 0x7fe228069350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe22c007fd0 tx=0x7fe22c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:46.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:46.944+0000 7fe223fff700 1 -- 172.21.15.7:0/719399240 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe234054490 con 0x7fe23c13b4c0 2023-10-07T07:57:47.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.151+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe23c0c83c0 con 0x7fe23c13b4c0 2023-10-07T07:57:47.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.152+0000 7fe223fff700 1 -- 172.21.15.7:0/719399240 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12175 (secure 0 0 0) 0x7fe234057ab0 con 0x7fe23c13b4c0 2023-10-07T07:57:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:47.153 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:46.677153+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:47.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe228066ea0 msgr2=0x7fe228069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe228066ea0 0x7fe228069350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe22c007fd0 tx=0x7fe22c009040 comp rx=0 tx=0).stop 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c13b4c0 msgr2=0x7fe23c0b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c13b4c0 0x7fe23c0b0e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe23c068770 tx=0x7fe234019000 comp rx=0 tx=0).stop 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 shutdown_connections 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe228066ea0 0x7fe228069350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c13b4c0 0x7fe23c0b0e40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe23c159310 0x7fe23c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 --2- 172.21.15.7:0/719399240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe23c1596e0 0x7fe23c13b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 >> 172.21.15.7:0/719399240 conn(0x7fe23c067ac0 msgr2=0x7fe23c154140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 shutdown_connections 2023-10-07T07:57:47.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.154+0000 7fe242480700 1 -- 172.21.15.7:0/719399240 wait complete. 2023-10-07T07:57:47.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T07:57:47.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:57:47.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.304+0000 7fe331c1d700 1 Processor -- start 2023-10-07T07:57:47.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.304+0000 7fe331c1d700 1 -- start start 2023-10-07T07:57:47.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe331c1d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe32c067240 0x7fe32c15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe331c1d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c067610 0x7fe32c153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe331c1d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe32c1539c0 0x7fe32c0aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe331c1d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c06a500 con 0x7fe32c1539c0 2023-10-07T07:57:47.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe331c1d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c06f320 con 0x7fe32c067240 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe331c1d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c153e00 con 0x7fe32c067610 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe32bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe32c1539c0 0x7fe32c0aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe32b7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe32c067240 0x7fe32c15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe32b7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe32c067240 0x7fe32c15a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:53328/0 (socket says 172.21.15.7:53328) 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe32b7fe700 1 -- 172.21.15.7:0/1994850668 learned_addr learned my addr 172.21.15.7:0/1994850668 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe32bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe32c1539c0 0x7fe32c0aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60180/0 (socket says 172.21.15.7:60180) 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.305+0000 7fe32affd700 1 --2- 172.21.15.7:0/1994850668 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c067610 0x7fe32c153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32bfff700 1 -- 172.21.15.7:0/1994850668 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c067610 msgr2=0x7fe32c153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32bfff700 1 --2- 172.21.15.7:0/1994850668 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c067610 0x7fe32c153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32bfff700 1 -- 172.21.15.7:0/1994850668 >> v2:172.21.15.172:3300/0 conn(0x7fe32c067240 msgr2=0x7fe32c15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32bfff700 1 --2- 172.21.15.7:0/1994850668 >> v2:172.21.15.172:3300/0 conn(0x7fe32c067240 0x7fe32c15a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32bfff700 1 -- 172.21.15.7:0/1994850668 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c06e7a0 con 0x7fe32c1539c0 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32b7fe700 1 --2- 172.21.15.7:0/1994850668 >> v2:172.21.15.172:3300/0 conn(0x7fe32c067240 0x7fe32c15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32affd700 1 --2- 172.21.15.7:0/1994850668 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c067610 0x7fe32c153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32bfff700 1 --2- 172.21.15.7:0/1994850668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c1539c0 0x7fe32c0aa980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe32c067ba0 tx=0x7fe324009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1d30a07a54eea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32a7fc700 1 -- 172.21.15.7:0/1994850668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32400e070 con 0x7fe32c1539c0 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32a7fc700 1 -- 172.21.15.7:0/1994850668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32400bab0 con 0x7fe32c1539c0 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe32a7fc700 1 -- 172.21.15.7:0/1994850668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32400be70 con 0x7fe32c1539c0 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe331c1d700 1 -- 172.21.15.7:0/1994850668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c1539c0 msgr2=0x7fe32c0aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe331c1d700 1 --2- 172.21.15.7:0/1994850668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c1539c0 0x7fe32c0aa980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe32c067ba0 tx=0x7fe324009780 comp rx=0 tx=0).stop 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe331c1d700 1 -- 172.21.15.7:0/1994850668 shutdown_connections 2023-10-07T07:57:47.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.306+0000 7fe331c1d700 1 --2- 172.21.15.7:0/1994850668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c1539c0 0x7fe32c0aa980 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.307+0000 7fe331c1d700 1 --2- 172.21.15.7:0/1994850668 >> v2:172.21.15.172:3300/0 conn(0x7fe32c067240 0x7fe32c15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.307+0000 7fe331c1d700 1 --2- 172.21.15.7:0/1994850668 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c067610 0x7fe32c153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.307+0000 7fe331c1d700 1 -- 172.21.15.7:0/1994850668 >> 172.21.15.7:0/1994850668 conn(0x7fe32c068100 msgr2=0x7fe32c068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:47.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.307+0000 7fe331c1d700 1 -- 172.21.15.7:0/1994850668 shutdown_connections 2023-10-07T07:57:47.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.307+0000 7fe331c1d700 1 -- 172.21.15.7:0/1994850668 wait complete. 2023-10-07T07:57:47.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.308+0000 7fe331c1d700 1 Processor -- start 2023-10-07T07:57:47.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.308+0000 7fe331c1d700 1 -- start start 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe331c1d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c067240 0x7fe32c14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe331c1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c067610 0x7fe32c0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe331c1d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c1539c0 0x7fe32c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe331c1d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c154cf0 con 0x7fe32c067610 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe331c1d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1501e0 con 0x7fe32c067240 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe331c1d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1540e0 con 0x7fe32c1539c0 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c067610 0x7fe32c0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32b7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c067240 0x7fe32c14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c067610 0x7fe32c0b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60188/0 (socket says 172.21.15.7:60188) 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32b7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c067240 0x7fe32c14fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:53330/0 (socket says 172.21.15.7:53330) 2023-10-07T07:57:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32affd700 1 -- 172.21.15.7:0/936045192 learned_addr learned my addr 172.21.15.7:0/936045192 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32bfff700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c1539c0 0x7fe32c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32b7fe700 1 -- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c1539c0 msgr2=0x7fe32c0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32b7fe700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c1539c0 0x7fe32c0cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32b7fe700 1 -- 172.21.15.7:0/936045192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c067610 msgr2=0x7fe32c0b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32b7fe700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c067610 0x7fe32c0b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32b7fe700 1 -- 172.21.15.7:0/936045192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0094f0 con 0x7fe32c067240 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.309+0000 7fe32b7fe700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c067240 0x7fe32c14fca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe31c006fd0 tx=0x7fe31c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.310+0000 7fe3137fe700 1 -- 172.21.15.7:0/936045192 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00c850 con 0x7fe32c067240 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.310+0000 7fe3137fe700 1 -- 172.21.15.7:0/936045192 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c00c9b0 con 0x7fe32c067240 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.310+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe320009420 con 0x7fe32c067240 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.310+0000 7fe3137fe700 1 -- 172.21.15.7:0/936045192 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00cd30 con 0x7fe32c067240 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.310+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe324009a20 con 0x7fe32c067240 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.311+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c154fd0 con 0x7fe32c067240 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.311+0000 7fe3137fe700 1 -- 172.21.15.7:0/936045192 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fe31c013680 con 0x7fe32c067240 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.311+0000 7fe3137fe700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe318066ea0 0x7fe318069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.312+0000 7fe3137fe700 1 -- 172.21.15.7:0/936045192 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe31c0842a0 con 0x7fe32c067240 2023-10-07T07:57:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.312+0000 7fe32affd700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe318066ea0 0x7fe318069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.312+0000 7fe32affd700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe318066ea0 0x7fe318069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe32c16ca20 tx=0x7fe3200091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.316+0000 7fe3137fe700 1 -- 172.21.15.7:0/936045192 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe31c0518a0 con 0x7fe32c067240 2023-10-07T07:57:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.522+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe32c04c3c0 con 0x7fe32c067240 2023-10-07T07:57:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.522+0000 7fe3137fe700 1 -- 172.21.15.7:0/936045192 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe31c054ec0 con 0x7fe32c067240 2023-10-07T07:57:47.523 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:57:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe318066ea0 msgr2=0x7fe318069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe318066ea0 0x7fe318069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe32c16ca20 tx=0x7fe3200091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c067240 msgr2=0x7fe32c14fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c067240 0x7fe32c14fca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe31c006fd0 tx=0x7fe31c009250 comp rx=0 tx=0).stop 2023-10-07T07:57:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 shutdown_connections 2023-10-07T07:57:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe318066ea0 0x7fe318069350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe32c1539c0 0x7fe32c0cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c067240 0x7fe32c14fca0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 --2- 172.21.15.7:0/936045192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe32c067610 0x7fe32c0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 >> 172.21.15.7:0/936045192 conn(0x7fe32c068100 msgr2=0x7fe32c0c2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 shutdown_connections 2023-10-07T07:57:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.525+0000 7fe331c1d700 1 -- 172.21.15.7:0/936045192 wait complete. 2023-10-07T07:57:47.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-07T07:57:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.670+0000 7f7eff4e0700 1 Processor -- start 2023-10-07T07:57:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.670+0000 7f7eff4e0700 1 -- start start 2023-10-07T07:57:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.671+0000 7f7eff4e0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef81592c0 0x7f7ef806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.671+0000 7f7eff4e0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef806bb70 con 0x7f7ef8159690 2023-10-07T07:57:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.671+0000 7f7eff4e0700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef80cda10 con 0x7f7ef80cb290 2023-10-07T07:57:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.671+0000 7f7eff4e0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef806af60 con 0x7f7ef81592c0 2023-10-07T07:57:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.671+0000 7f7efca7b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7ef8159690 0x7f7ef80cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38616/0 (socket says 172.21.15.7:38616) 2023-10-07T07:57:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.671+0000 7f7efca7b700 1 -- 172.21.15.7:0/3525437058 learned_addr learned my addr 172.21.15.7:0/3525437058 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.671+0000 7f7efd27c700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef81592c0 0x7f7ef806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.671+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894146031 0 0) 0x7f7ef806bb70 con 0x7f7ef8159690 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.671+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ee4003570 con 0x7f7ef8159690 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937457712 0 0) 0x7f7ef80cda10 con 0x7f7ef80cb290 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ef806bb70 con 0x7f7ef80cb290 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3886114659 0 0) 0x7f7ee4003570 con 0x7f7ef8159690 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef81592c0 msgr2=0x7f7ef806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef81592c0 0x7f7ef806a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 >> v1:172.21.15.172:6789/0 conn(0x7f7ef80cb290 legacy=0x7f7ef80a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef806b240 con 0x7f7ef8159690 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f7ef4001470 con 0x7f7ef8159690 2023-10-07T07:57:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 >> v1:172.21.15.7:6789/0 conn(0x7f7ef8159690 legacy=0x7f7ef80cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ee4003c20 0x7f7ee4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ee40051c0 0x7f7ee400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef806af60 con 0x7f7ee4004810 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee400c130 con 0x7f7ee40051c0 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee400c2c0 con 0x7f7ee4003c20 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7efd27c700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef81592c0 0x7f7ef806a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7efca7b700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7efda7d700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ee40051c0 0x7f7ee400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7efd27c700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ee4003c20 0x7f7ee4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7efca7b700 1 -- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ee4003c20 msgr2=0x7f7ee4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7efca7b700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ee4003c20 0x7f7ee4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7efca7b700 1 -- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ee40051c0 msgr2=0x7f7ee400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7efca7b700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ee40051c0 0x7f7ee400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.672+0000 7f7efca7b700 1 -- 172.21.15.7:0/3525437058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee4003570 con 0x7f7ee4004810 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7efca7b700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c210 tx=0x7f7ef80cb9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992b7941859ffa17 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400cc10 con 0x7f7ee4004810 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef400cd70 con 0x7f7ee4004810 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7eeffff700 1 -- 172.21.15.7:0/3525437058 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4005b70 con 0x7f7ee4004810 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7eff4e0700 1 -- 172.21.15.7:0/3525437058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 msgr2=0x7f7ee4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7eff4e0700 1 --2- 172.21.15.7:0/3525437058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c210 tx=0x7f7ef80cb9e0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7efda7d700 1 -- 172.21.15.7:0/3525437058 reap_dead start 2023-10-07T07:57:47.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7eff4e0700 1 -- 172.21.15.7:0/3525437058 shutdown_connections 2023-10-07T07:57:47.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7eff4e0700 1 -- 172.21.15.7:0/3525437058 >> 172.21.15.7:0/3525437058 conn(0x7f7ef8068bd0 msgr2=0x7f7ef80ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:47.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7eff4e0700 1 -- 172.21.15.7:0/3525437058 shutdown_connections 2023-10-07T07:57:47.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.673+0000 7f7eff4e0700 1 -- 172.21.15.7:0/3525437058 wait complete. 2023-10-07T07:57:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.675+0000 7f7eff4e0700 1 Processor -- start 2023-10-07T07:57:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.675+0000 7f7eff4e0700 1 -- start start 2023-10-07T07:57:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eff4e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ef80a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eff4e0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef80acf40 0x7f7ef81561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eff4e0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef816a100 0x7f7ef80aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eff4e0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8156700 con 0x7f7ee4004810 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eff4e0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef80acab0 con 0x7f7ef816a100 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eff4e0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef80aa340 con 0x7f7ef80acf40 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efd27c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ef80a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efd27c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ef80a9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60208/0 (socket says 172.21.15.7:60208) 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efd27c700 1 -- 172.21.15.7:0/2192873243 learned_addr learned my addr 172.21.15.7:0/2192873243 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efda7d700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef816a100 0x7f7ef80aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efca7b700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef80acf40 0x7f7ef81561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efd27c700 1 -- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef80acf40 msgr2=0x7f7ef81561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efd27c700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef80acf40 0x7f7ef81561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efd27c700 1 -- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef816a100 msgr2=0x7f7ef80aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efd27c700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef816a100 0x7f7ef80aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efd27c700 1 -- 172.21.15.7:0/2192873243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef80ac180 con 0x7f7ee4004810 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7efd27c700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ef80a9d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8011d20 tx=0x7f7ee80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eed7fa700 1 -- 172.21.15.7:0/2192873243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800fa10 con 0x7f7ee4004810 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eed7fa700 1 -- 172.21.15.7:0/2192873243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee800fb70 con 0x7f7ee4004810 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eed7fa700 1 -- 172.21.15.7:0/2192873243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee801d480 con 0x7f7ee4004810 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.676+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef80ac460 con 0x7f7ee4004810 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.677+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef816a5a0 con 0x7f7ee4004810 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.677+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef80716c0 con 0x7f7ee4004810 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.678+0000 7f7eed7fa700 1 -- 172.21.15.7:0/2192873243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f7ee801da80 con 0x7f7ee4004810 2023-10-07T07:57:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.678+0000 7f7eed7fa700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7edc066ea0 0x7f7edc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.678+0000 7f7eed7fa700 1 -- 172.21.15.7:0/2192873243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7ee8018070 con 0x7f7ee4004810 2023-10-07T07:57:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.679+0000 7f7efca7b700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7edc066ea0 0x7f7edc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.679+0000 7f7efca7b700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7edc066ea0 0x7f7edc069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c210 tx=0x7f7ef4015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.683+0000 7f7eed7fa700 1 -- 172.21.15.7:0/2192873243 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ee8052710 con 0x7f7ee4004810 2023-10-07T07:57:47.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.888+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7ef80acd90 con 0x7f7ee4004810 2023-10-07T07:57:47.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.888+0000 7f7eed7fa700 1 -- 172.21.15.7:0/2192873243 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7ee8055d30 con 0x7f7ee4004810 2023-10-07T07:57:47.889 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:47.890 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2023-10-07T07:57:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:57:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2023-10-07T07:57:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2023-10-07T07:57:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:57:47.890 INFO:teuthology.orchestra.run.smithi007.stdout:] 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.890+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7edc066ea0 msgr2=0x7f7edc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.890+0000 7f7eff4e0700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7edc066ea0 0x7f7edc069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c210 tx=0x7f7ef4015000 comp rx=0 tx=0).stop 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.890+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 msgr2=0x7f7ef80a9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.890+0000 7f7eff4e0700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ef80a9d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8011d20 tx=0x7f7ee80091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.890+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 shutdown_connections 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.890+0000 7f7eff4e0700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7edc066ea0 0x7f7edc069350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.890+0000 7f7eff4e0700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef816a100 0x7f7ef80aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.890+0000 7f7eff4e0700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee4004810 0x7f7ef80a9d60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.890+0000 7f7eff4e0700 1 --2- 172.21.15.7:0/2192873243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ef80acf40 0x7f7ef81561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.891+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 >> 172.21.15.7:0/2192873243 conn(0x7f7ef8068bd0 msgr2=0x7f7ef81572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.891+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 shutdown_connections 2023-10-07T07:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:47.891+0000 7f7eff4e0700 1 -- 172.21.15.7:0/2192873243 wait complete. 2023-10-07T07:57:47.905 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-07T07:57:47.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-07T07:57:48.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.040+0000 7fcff3abd700 1 Processor -- start 2023-10-07T07:57:48.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.040+0000 7fcff3abd700 1 -- start start 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff3abd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec06bc70 0x7fcfec0ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff3abd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec0cc040 con 0x7fcfec15b8f0 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff3abd700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec0cbd10 con 0x7fcfec140bf0 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff3abd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec15bd40 con 0x7fcfec06bc70 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff1058700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcfec15b8f0 0x7fcfec06b560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38630/0 (socket says 172.21.15.7:38630) 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff1058700 1 -- 172.21.15.7:0/1327147649 learned_addr learned my addr 172.21.15.7:0/1327147649 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff205a700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec06bc70 0x7fcfec0ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638596664 0 0) 0x7fcfec0cc040 con 0x7fcfec15b8f0 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfd8003570 con 0x7fcfec15b8f0 2023-10-07T07:57:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266867745 0 0) 0x7fcfec0cbd10 con 0x7fcfec140bf0 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfec0cc040 con 0x7fcfec140bf0 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4230255424 0 0) 0x7fcfd8003570 con 0x7fcfec15b8f0 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec06bc70 msgr2=0x7fcfec0ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.041+0000 7fcff0857700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec06bc70 0x7fcfec0ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 >> v1:172.21.15.172:6789/0 conn(0x7fcfec140bf0 legacy=0x7fcfec0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0c8780 con 0x7fcfec15b8f0 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7fcfe8001470 con 0x7fcfec15b8f0 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 >> v1:172.21.15.7:6789/0 conn(0x7fcfec15b8f0 legacy=0x7fcfec06b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec06bc70 0x7fcfd8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80041d0 0x7fcfd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd8004d50 0x7fcfd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec15bd40 con 0x7fcfd8004d50 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd800bbf0 con 0x7fcfd80041d0 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd800bdb0 con 0x7fcfec06bc70 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff205a700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd8004d50 0x7fcfd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff1058700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80041d0 0x7fcfd8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff1859700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec06bc70 0x7fcfd8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff205a700 1 -- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec06bc70 msgr2=0x7fcfd8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff205a700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec06bc70 0x7fcfd8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff205a700 1 -- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80041d0 msgr2=0x7fcfd8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff205a700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd80041d0 0x7fcfd8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff205a700 1 -- 172.21.15.7:0/1327147649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8003570 con 0x7fcfd8004d50 2023-10-07T07:57:48.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff205a700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd8004d50 0x7fcfd80051c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0ce9a0 tx=0x7fcfe400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45639dcc08c9da6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe400ea60 con 0x7fcfd8004d50 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe400f040 con 0x7fcfd8004d50 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.042+0000 7fcff0857700 1 -- 172.21.15.7:0/1327147649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe400ee10 con 0x7fcfd8004d50 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.043+0000 7fcff3abd700 1 -- 172.21.15.7:0/1327147649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd8004d50 msgr2=0x7fcfd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.043+0000 7fcff3abd700 1 --2- 172.21.15.7:0/1327147649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd8004d50 0x7fcfd80051c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0ce9a0 tx=0x7fcfe400b040 comp rx=0 tx=0).stop 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.043+0000 7fcff205a700 1 -- 172.21.15.7:0/1327147649 reap_dead start 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.043+0000 7fcff3abd700 1 -- 172.21.15.7:0/1327147649 shutdown_connections 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.043+0000 7fcff3abd700 1 -- 172.21.15.7:0/1327147649 >> 172.21.15.7:0/1327147649 conn(0x7fcfec067ac0 msgr2=0x7fcfec0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.043+0000 7fcff3abd700 1 -- 172.21.15.7:0/1327147649 shutdown_connections 2023-10-07T07:57:48.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.043+0000 7fcff3abd700 1 -- 172.21.15.7:0/1327147649 wait complete. 2023-10-07T07:57:48.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.044+0000 7fcff3abd700 1 Processor -- start 2023-10-07T07:57:48.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.044+0000 7fcff3abd700 1 -- start start 2023-10-07T07:57:48.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff3abd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec156a60 0x7fcfec1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff3abd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfec0b3910 0x7fcfec13f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff3abd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfec13fe40 0x7fcfec0b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff3abd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec1402c0 con 0x7fcfec0b3910 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff3abd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec158a70 con 0x7fcfec13fe40 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff3abd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec157e60 con 0x7fcfec156a60 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff1058700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfec0b3910 0x7fcfec13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff1058700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfec0b3910 0x7fcfec13f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60234/0 (socket says 172.21.15.7:60234) 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff1058700 1 -- 172.21.15.7:0/2887402056 learned_addr learned my addr 172.21.15.7:0/2887402056 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff205a700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfec13fe40 0x7fcfec0b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff1859700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec156a60 0x7fcfec1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.045+0000 7fcff1058700 1 -- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec156a60 msgr2=0x7fcfec1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcff1058700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec156a60 0x7fcfec1574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcff1058700 1 -- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfec13fe40 msgr2=0x7fcfec0b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcff1058700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfec13fe40 0x7fcfec0b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcff1058700 1 -- 172.21.15.7:0/2887402056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec158140 con 0x7fcfec0b3910 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcff1859700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec156a60 0x7fcfec1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcff1058700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfec0b3910 0x7fcfec13f900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8004fd0 tx=0x7fcfe8001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcfe1ffb700 1 -- 172.21.15.7:0/2887402056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8000f50 con 0x7fcfec0b3910 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcfe1ffb700 1 -- 172.21.15.7:0/2887402056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe800f8d0 con 0x7fcfec0b3910 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec158420 con 0x7fcfec0b3910 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcfe1ffb700 1 -- 172.21.15.7:0/2887402056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800fc90 con 0x7fcfec0b3910 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.046+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec158f00 con 0x7fcfec0b3910 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.047+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec158c20 con 0x7fcfec0b3910 2023-10-07T07:57:48.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.048+0000 7fcfe1ffb700 1 -- 172.21.15.7:0/2887402056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fcfe8013600 con 0x7fcfec0b3910 2023-10-07T07:57:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.048+0000 7fcfe1ffb700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfd0066ea0 0x7fcfd0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.048+0000 7fcfe1ffb700 1 -- 172.21.15.7:0/2887402056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcfe8084220 con 0x7fcfec0b3910 2023-10-07T07:57:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.048+0000 7fcff1859700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfd0066ea0 0x7fcfd0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.049+0000 7fcff1859700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfd0066ea0 0x7fcfd0069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc001130 tx=0x7fcfdc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.053+0000 7fcfe1ffb700 1 -- 172.21.15.7:0/2887402056 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfec158c20 con 0x7fcfec0b3910 2023-10-07T07:57:48.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.253+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcfec158be0 con 0x7fcfec0b3910 2023-10-07T07:57:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.695+0000 7fcfe1ffb700 1 -- 172.21.15.7:0/2887402056 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfe8051c90 con 0x7fcfec0b3910 2023-10-07T07:57:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.700+0000 7fcfe1ffb700 1 -- 172.21.15.7:0/2887402056 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcfe8070360 con 0x7fcfec0b3910 2023-10-07T07:57:48.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfd0066ea0 msgr2=0x7fcfd0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfd0066ea0 0x7fcfd0069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc001130 tx=0x7fcfdc009040 comp rx=0 tx=0).stop 2023-10-07T07:57:48.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfec0b3910 msgr2=0x7fcfec13f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfec0b3910 0x7fcfec13f900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8004fd0 tx=0x7fcfe8001d40 comp rx=0 tx=0).stop 2023-10-07T07:57:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 shutdown_connections 2023-10-07T07:57:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfd0066ea0 0x7fcfd0069350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfec13fe40 0x7fcfec0b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfec156a60 0x7fcfec1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 --2- 172.21.15.7:0/2887402056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfec0b3910 0x7fcfec13f900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.705+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 >> 172.21.15.7:0/2887402056 conn(0x7fcfec067ac0 msgr2=0x7fcfec06b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.706+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 shutdown_connections 2023-10-07T07:57:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.706+0000 7fcff3abd700 1 -- 172.21.15.7:0/2887402056 wait complete. 2023-10-07T07:57:48.708 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2023-10-07T07:57:48.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-07T07:57:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.863+0000 7f1d21f51700 1 Processor -- start 2023-10-07T07:57:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.863+0000 7f1d21f51700 1 -- start start 2023-10-07T07:57:48.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.864+0000 7f1d21f51700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d1c15c920 0x7f1d1c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.864+0000 7f1d21f51700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1d1c15ccf0 0x7f1d1c06c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.864+0000 7f1d21f51700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1c06c600 con 0x7f1d1c0c6d70 2023-10-07T07:57:48.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.864+0000 7f1d21f51700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c0cb3c0 con 0x7f1d1c15c920 2023-10-07T07:57:48.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.864+0000 7f1d21f51700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c0cab00 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.864+0000 7f1d1b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d1c15c920 0x7f1d1c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.864+0000 7f1d1b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d1c15c920 0x7f1d1c0ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60248/0 (socket says 172.21.15.7:60248) 2023-10-07T07:57:48.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.864+0000 7f1d1b7fe700 1 -- 172.21.15.7:0/280528916 learned_addr learned my addr 172.21.15.7:0/280528916 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:48.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1affd700 1 --2- 172.21.15.7:0/280528916 >> v2:172.21.15.172:3300/0 conn(0x7f1d1c15ccf0 0x7f1d1c06c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1b7fe700 1 -- 172.21.15.7:0/280528916 >> v2:172.21.15.172:3300/0 conn(0x7f1d1c15ccf0 msgr2=0x7f1d1c06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1b7fe700 1 --2- 172.21.15.7:0/280528916 >> v2:172.21.15.172:3300/0 conn(0x7f1d1c15ccf0 0x7f1d1c06c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1b7fe700 1 -- 172.21.15.7:0/280528916 >> v1:172.21.15.172:6789/0 conn(0x7f1d1c0c6d70 legacy=0x7f1d1c068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1b7fe700 1 -- 172.21.15.7:0/280528916 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c06a7a0 con 0x7f1d1c15c920 2023-10-07T07:57:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1affd700 1 --2- 172.21.15.7:0/280528916 >> v2:172.21.15.172:3300/0 conn(0x7f1d1c15ccf0 0x7f1d1c06c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:57:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1b7fe700 1 --2- 172.21.15.7:0/280528916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15c920 0x7f1d1c0ca520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0c3760 tx=0x7f1d0c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d087cf64c6660896 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1a7fc700 1 -- 172.21.15.7:0/280528916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c005c10 con 0x7f1d1c15c920 2023-10-07T07:57:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1a7fc700 1 -- 172.21.15.7:0/280528916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c005d70 con 0x7f1d1c15c920 2023-10-07T07:57:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.865+0000 7f1d1a7fc700 1 -- 172.21.15.7:0/280528916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c00b8a0 con 0x7f1d1c15c920 2023-10-07T07:57:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.866+0000 7f1d21f51700 1 -- 172.21.15.7:0/280528916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15c920 msgr2=0x7f1d1c0ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.866+0000 7f1d21f51700 1 --2- 172.21.15.7:0/280528916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15c920 0x7f1d1c0ca520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0c3760 tx=0x7f1d0c009220 comp rx=0 tx=0).stop 2023-10-07T07:57:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.866+0000 7f1d21f51700 1 -- 172.21.15.7:0/280528916 shutdown_connections 2023-10-07T07:57:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.866+0000 7f1d21f51700 1 --2- 172.21.15.7:0/280528916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15c920 0x7f1d1c0ca520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.866+0000 7f1d21f51700 1 --2- 172.21.15.7:0/280528916 >> v2:172.21.15.172:3300/0 conn(0x7f1d1c15ccf0 0x7f1d1c06c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.866+0000 7f1d21f51700 1 -- 172.21.15.7:0/280528916 >> 172.21.15.7:0/280528916 conn(0x7f1d1c0c9c30 msgr2=0x7f1d1c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.866+0000 7f1d21f51700 1 -- 172.21.15.7:0/280528916 shutdown_connections 2023-10-07T07:57:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.866+0000 7f1d21f51700 1 -- 172.21.15.7:0/280528916 wait complete. 2023-10-07T07:57:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.867+0000 7f1d21f51700 1 Processor -- start 2023-10-07T07:57:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.867+0000 7f1d21f51700 1 -- start start 2023-10-07T07:57:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d21f51700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d1c0c6d70 0x7f1d1c14a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d21f51700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d1c15c920 0x7f1d1c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d21f51700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15ccf0 0x7f1d1c16a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d21f51700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c1569a0 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d21f51700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c15c1b0 con 0x7f1d1c0c6d70 2023-10-07T07:57:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d21f51700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c16aa20 con 0x7f1d1c15c920 2023-10-07T07:57:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d1bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15ccf0 0x7f1d1c16a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d1bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15ccf0 0x7f1d1c16a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60262/0 (socket says 172.21.15.7:60262) 2023-10-07T07:57:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d1bfff700 1 -- 172.21.15.7:0/2965092742 learned_addr learned my addr 172.21.15.7:0/2965092742 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d1b7fe700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d1c0c6d70 0x7f1d1c14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d1affd700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d1c15c920 0x7f1d1c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d1bfff700 1 -- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d1c15c920 msgr2=0x7f1d1c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.868+0000 7f1d1bfff700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d1c15c920 0x7f1d1c15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.869+0000 7f1d1bfff700 1 -- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d1c0c6d70 msgr2=0x7f1d1c14a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.869+0000 7f1d1bfff700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d1c0c6d70 0x7f1d1c14a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.869+0000 7f1d1bfff700 1 -- 172.21.15.7:0/2965092742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c009480 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.869+0000 7f1d1bfff700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15ccf0 0x7f1d1c16a4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c840 tx=0x7f1d14001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.869+0000 7f1d137fe700 1 -- 172.21.15.7:0/2965092742 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14012630 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.869+0000 7f1d137fe700 1 -- 172.21.15.7:0/2965092742 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d140127b0 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.869+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c15b5a0 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.869+0000 7f1d137fe700 1 -- 172.21.15.7:0/2965092742 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14012b70 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.869+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c15bb60 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.870+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c063620 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.870+0000 7f1d137fe700 1 -- 172.21.15.7:0/2965092742 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d1401a3a0 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.871+0000 7f1d137fe700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1cfc066f30 0x7f1cfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.871+0000 7f1d137fe700 1 -- 172.21.15.7:0/2965092742 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f1d14084270 con 0x7f1d1c15ccf0 2023-10-07T07:57:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.871+0000 7f1d1b7fe700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1cfc066f30 0x7f1cfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.871+0000 7f1d1b7fe700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1cfc066f30 0x7f1cfc0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c007fd0 tx=0x7f1d0c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:48.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:48.875+0000 7f1d137fe700 1 -- 172.21.15.7:0/2965092742 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d140516c0 con 0x7f1d1c15ccf0 2023-10-07T07:57:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.079+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1d1c06c600 con 0x7f1d1c15ccf0 2023-10-07T07:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.704+0000 7f1d137fe700 1 -- 172.21.15.7:0/2965092742 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1d1c06c600 con 0x7f1d1c15ccf0 2023-10-07T07:57:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1cfc066f30 msgr2=0x7f1cfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1cfc066f30 0x7f1cfc0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c007fd0 tx=0x7f1d0c016000 comp rx=0 tx=0).stop 2023-10-07T07:57:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15ccf0 msgr2=0x7f1d1c16a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15ccf0 0x7f1d1c16a4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c840 tx=0x7f1d14001670 comp rx=0 tx=0).stop 2023-10-07T07:57:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 shutdown_connections 2023-10-07T07:57:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1cfc066f30 0x7f1cfc0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c15ccf0 0x7f1d1c16a4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d1c0c6d70 0x7f1d1c14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 --2- 172.21.15.7:0/2965092742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d1c15c920 0x7f1d1c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 >> 172.21.15.7:0/2965092742 conn(0x7f1d1c0c9c30 msgr2=0x7f1d1c1584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 shutdown_connections 2023-10-07T07:57:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.707+0000 7f1d21f51700 1 -- 172.21.15.7:0/2965092742 wait complete. 2023-10-07T07:57:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2023-10-07T07:57:49.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-07T07:57:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.872+0000 7fbc5434f700 1 Processor -- start 2023-10-07T07:57:49.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.872+0000 7fbc5434f700 1 -- start start 2023-10-07T07:57:49.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.873+0000 7fbc5434f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc4c0cc410 0x7fbc4c0aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:49.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.873+0000 7fbc5434f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4c06f320 con 0x7fbc4c067240 2023-10-07T07:57:49.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.873+0000 7fbc5434f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4c06e7a0 con 0x7fbc4c067610 2023-10-07T07:57:49.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.873+0000 7fbc5434f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c06a500 con 0x7fbc4c0cc410 2023-10-07T07:57:49.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.873+0000 7fbc528ec700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc4c0cc410 0x7fbc4c0aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:49.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.873+0000 7fbc520eb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbc4c067240 0x7fbc4c15a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35894/0 (socket says 172.21.15.7:35894) 2023-10-07T07:57:49.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.873+0000 7fbc520eb700 1 -- 172.21.15.7:0/4054188747 learned_addr learned my addr 172.21.15.7:0/4054188747 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.874+0000 7fbc528ec700 1 -- 172.21.15.7:0/4054188747 >> v1:172.21.15.172:6789/0 conn(0x7fbc4c067610 legacy=0x7fbc4c153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.874+0000 7fbc528ec700 1 -- 172.21.15.7:0/4054188747 >> v1:172.21.15.7:6789/0 conn(0x7fbc4c067240 legacy=0x7fbc4c15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.874+0000 7fbc528ec700 1 -- 172.21.15.7:0/4054188747 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c06ea80 con 0x7fbc4c0cc410 2023-10-07T07:57:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.874+0000 7fbc528ec700 1 --2- 172.21.15.7:0/4054188747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c0cc410 0x7fbc4c0aa980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009e60 tx=0x7fbc44009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150006401534e5b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:49.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.874+0000 7fbc510e9700 1 -- 172.21.15.7:0/4054188747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400e070 con 0x7fbc4c0cc410 2023-10-07T07:57:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.874+0000 7fbc510e9700 1 -- 172.21.15.7:0/4054188747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4400bc30 con 0x7fbc4c0cc410 2023-10-07T07:57:49.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.874+0000 7fbc510e9700 1 -- 172.21.15.7:0/4054188747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44012430 con 0x7fbc4c0cc410 2023-10-07T07:57:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.875+0000 7fbc5434f700 1 -- 172.21.15.7:0/4054188747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c0cc410 msgr2=0x7fbc4c0aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.875+0000 7fbc5434f700 1 --2- 172.21.15.7:0/4054188747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c0cc410 0x7fbc4c0aa980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009e60 tx=0x7fbc44009780 comp rx=0 tx=0).stop 2023-10-07T07:57:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.875+0000 7fbc5434f700 1 -- 172.21.15.7:0/4054188747 shutdown_connections 2023-10-07T07:57:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.875+0000 7fbc5434f700 1 --2- 172.21.15.7:0/4054188747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c0cc410 0x7fbc4c0aa980 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.875+0000 7fbc5434f700 1 -- 172.21.15.7:0/4054188747 >> 172.21.15.7:0/4054188747 conn(0x7fbc4c068100 msgr2=0x7fbc4c068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.875+0000 7fbc5434f700 1 -- 172.21.15.7:0/4054188747 shutdown_connections 2023-10-07T07:57:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.875+0000 7fbc5434f700 1 -- 172.21.15.7:0/4054188747 wait complete. 2023-10-07T07:57:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.876+0000 7fbc5434f700 1 Processor -- start 2023-10-07T07:57:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.876+0000 7fbc5434f700 1 -- start start 2023-10-07T07:57:49.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc5434f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc4c067240 0x7fbc4c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:49.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc5434f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc4c067610 0x7fbc4c0c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:49.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc5434f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c15e970 0x7fbc4c153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:49.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc5434f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0afd40 con 0x7fbc4c15e970 2023-10-07T07:57:49.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc5434f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c15d820 con 0x7fbc4c067240 2023-10-07T07:57:49.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc5434f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0b0880 con 0x7fbc4c067610 2023-10-07T07:57:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc528ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c15e970 0x7fbc4c153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc528ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c15e970 0x7fbc4c153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33392/0 (socket says 172.21.15.7:33392) 2023-10-07T07:57:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc528ec700 1 -- 172.21.15.7:0/273107947 learned_addr learned my addr 172.21.15.7:0/273107947 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc520eb700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc4c067240 0x7fbc4c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc518ea700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc4c067610 0x7fbc4c0c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc528ec700 1 -- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc4c067610 msgr2=0x7fbc4c0c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc528ec700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc4c067610 0x7fbc4c0c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc528ec700 1 -- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc4c067240 msgr2=0x7fbc4c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc528ec700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc4c067240 0x7fbc4c15d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc528ec700 1 -- 172.21.15.7:0/273107947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44009a20 con 0x7fbc4c15e970 2023-10-07T07:57:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.877+0000 7fbc528ec700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c15e970 0x7fbc4c153b90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbc440097c0 tx=0x7fbc44019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.878+0000 7fbc3e7fc700 1 -- 172.21.15.7:0/273107947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44012e10 con 0x7fbc4c15e970 2023-10-07T07:57:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.878+0000 7fbc3e7fc700 1 -- 172.21.15.7:0/273107947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4401e630 con 0x7fbc4c15e970 2023-10-07T07:57:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.878+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c0c45d0 con 0x7fbc4c15e970 2023-10-07T07:57:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.878+0000 7fbc3e7fc700 1 -- 172.21.15.7:0/273107947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4401e9f0 con 0x7fbc4c15e970 2023-10-07T07:57:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.878+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c0b0230 con 0x7fbc4c15e970 2023-10-07T07:57:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.879+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4c04c3c0 con 0x7fbc4c15e970 2023-10-07T07:57:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.879+0000 7fbc3e7fc700 1 -- 172.21.15.7:0/273107947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc44024070 con 0x7fbc4c15e970 2023-10-07T07:57:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.880+0000 7fbc3e7fc700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbc30066f30 0x7fbc300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.880+0000 7fbc3e7fc700 1 -- 172.21.15.7:0/273107947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fbc44086e00 con 0x7fbc4c15e970 2023-10-07T07:57:49.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.880+0000 7fbc520eb700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbc30066f30 0x7fbc300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.880+0000 7fbc520eb700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbc30066f30 0x7fbc300693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc40007130 tx=0x7fbc40007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:49.885+0000 7fbc3e7fc700 1 -- 172.21.15.7:0/273107947 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc44054030 con 0x7fbc4c15e970 2023-10-07T07:57:50.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.137+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbc4c068fa0 con 0x7fbc4c15e970 2023-10-07T07:57:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.739+0000 7fbc3e7fc700 1 -- 172.21.15.7:0/273107947 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbc44057650 con 0x7fbc4c15e970 2023-10-07T07:57:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.743+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbc30066f30 msgr2=0x7fbc300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.743+0000 7fbc5434f700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbc30066f30 0x7fbc300693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc40007130 tx=0x7fbc40007eb0 comp rx=0 tx=0).stop 2023-10-07T07:57:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.743+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c15e970 msgr2=0x7fbc4c153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.743+0000 7fbc5434f700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c15e970 0x7fbc4c153b90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbc440097c0 tx=0x7fbc44019000 comp rx=0 tx=0).stop 2023-10-07T07:57:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.744+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 shutdown_connections 2023-10-07T07:57:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.744+0000 7fbc5434f700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbc30066f30 0x7fbc300693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.744+0000 7fbc5434f700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc4c15e970 0x7fbc4c153b90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.744+0000 7fbc5434f700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc4c067240 0x7fbc4c15d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.744+0000 7fbc5434f700 1 --2- 172.21.15.7:0/273107947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc4c067610 0x7fbc4c0c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.744+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 >> 172.21.15.7:0/273107947 conn(0x7fbc4c068100 msgr2=0x7fbc4c1579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.745+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 shutdown_connections 2023-10-07T07:57:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.745+0000 7fbc5434f700 1 -- 172.21.15.7:0/273107947 wait complete. 2023-10-07T07:57:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2023-10-07T07:57:50.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.894+0000 7f9086c9f700 1 Processor -- start 2023-10-07T07:57:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.894+0000 7f9086c9f700 1 -- start start 2023-10-07T07:57:50.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.895+0000 7f9086c9f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9080159690 0x7f90800cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:50.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.895+0000 7f9086c9f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908006bb70 con 0x7f90801592c0 2023-10-07T07:57:50.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.895+0000 7f9086c9f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90800cda10 con 0x7f90800cb0c0 2023-10-07T07:57:50.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.895+0000 7f9086c9f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908006af60 con 0x7f9080159690 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.895+0000 7f9084a3b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f90801592c0 0x7f908006a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35902/0 (socket says 172.21.15.7:35902) 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.895+0000 7f9084a3b700 1 -- 172.21.15.7:0/2740992906 learned_addr learned my addr 172.21.15.7:0/2740992906 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907ffff700 1 --2- 172.21.15.7:0/2740992906 >> v2:172.21.15.172:3300/0 conn(0x7f9080159690 0x7f90800cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907f7fe700 1 -- 172.21.15.7:0/2740992906 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124809011 0 0) 0x7f908006bb70 con 0x7f90801592c0 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907f7fe700 1 -- 172.21.15.7:0/2740992906 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9070003570 con 0x7f90801592c0 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907f7fe700 1 -- 172.21.15.7:0/2740992906 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669810616 0 0) 0x7f90800cda10 con 0x7f90800cb0c0 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907f7fe700 1 -- 172.21.15.7:0/2740992906 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f908006bb70 con 0x7f90800cb0c0 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907ffff700 1 -- 172.21.15.7:0/2740992906 >> v1:172.21.15.172:6789/0 conn(0x7f90800cb0c0 legacy=0x7f90800a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907ffff700 1 -- 172.21.15.7:0/2740992906 >> v1:172.21.15.7:6789/0 conn(0x7f90801592c0 legacy=0x7f908006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907ffff700 1 -- 172.21.15.7:0/2740992906 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908006b240 con 0x7f9080159690 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907ffff700 1 --2- 172.21.15.7:0/2740992906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9080159690 0x7f90800cab80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f906c0098e0 tx=0x7f906c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a12e19f5ae9ed3fc server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907f7fe700 1 -- 172.21.15.7:0/2740992906 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c011630 con 0x7f9080159690 2023-10-07T07:57:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907f7fe700 1 -- 172.21.15.7:0/2740992906 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c011790 con 0x7f9080159690 2023-10-07T07:57:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.896+0000 7f907f7fe700 1 -- 172.21.15.7:0/2740992906 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c011b30 con 0x7f9080159690 2023-10-07T07:57:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.897+0000 7f9086c9f700 1 -- 172.21.15.7:0/2740992906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9080159690 msgr2=0x7f90800cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.897+0000 7f9086c9f700 1 --2- 172.21.15.7:0/2740992906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9080159690 0x7f90800cab80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f906c0098e0 tx=0x7f906c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.897+0000 7f9086c9f700 1 -- 172.21.15.7:0/2740992906 shutdown_connections 2023-10-07T07:57:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.897+0000 7f9086c9f700 1 --2- 172.21.15.7:0/2740992906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9080159690 0x7f90800cab80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.897+0000 7f9086c9f700 1 -- 172.21.15.7:0/2740992906 >> 172.21.15.7:0/2740992906 conn(0x7f9080068bd0 msgr2=0x7f90800ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.897+0000 7f9086c9f700 1 -- 172.21.15.7:0/2740992906 shutdown_connections 2023-10-07T07:57:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.897+0000 7f9086c9f700 1 -- 172.21.15.7:0/2740992906 wait complete. 2023-10-07T07:57:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.898+0000 7f9086c9f700 1 Processor -- start 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.898+0000 7f9086c9f700 1 -- start start 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f9086c9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90800cb0c0 0x7f90800ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f9086c9f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90801592c0 0x7f90801694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f9086c9f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9080159690 0x7f908015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f9086c9f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908016a500 con 0x7f90800cb0c0 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f9086c9f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908015de80 con 0x7f90801592c0 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f9086c9f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90801699e0 con 0x7f9080159690 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f9084a3b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90800cb0c0 0x7f90800ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f9084a3b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90800cb0c0 0x7f90800ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33404/0 (socket says 172.21.15.7:33404) 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f9084a3b700 1 -- 172.21.15.7:0/1498239053 learned_addr learned my addr 172.21.15.7:0/1498239053 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f907ffff700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90801592c0 0x7f90801694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.899+0000 7f908523c700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9080159690 0x7f908015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f9084a3b700 1 -- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9080159690 msgr2=0x7f908015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f9084a3b700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9080159690 0x7f908015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f9084a3b700 1 -- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90801592c0 msgr2=0x7f90801694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f9084a3b700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90801592c0 0x7f90801694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f9084a3b700 1 -- 172.21.15.7:0/1498239053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c009420 con 0x7f90800cb0c0 2023-10-07T07:57:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f9084a3b700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90800cb0c0 0x7f90800ae430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f907400c7c0 tx=0x7f9074007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f907cff9700 1 -- 172.21.15.7:0/1498239053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074005b30 con 0x7f90800cb0c0 2023-10-07T07:57:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f907cff9700 1 -- 172.21.15.7:0/1498239053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074005c90 con 0x7f90800cb0c0 2023-10-07T07:57:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908015db50 con 0x7f90800cb0c0 2023-10-07T07:57:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f907cff9700 1 -- 172.21.15.7:0/1498239053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074007300 con 0x7f90800cb0c0 2023-10-07T07:57:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.900+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9080169fc0 con 0x7f90800cb0c0 2023-10-07T07:57:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.901+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90800716c0 con 0x7f90800cb0c0 2023-10-07T07:57:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.901+0000 7f907cff9700 1 -- 172.21.15.7:0/1498239053 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f907401b3b0 con 0x7f90800cb0c0 2023-10-07T07:57:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.902+0000 7f907cff9700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9064066f30 0x7f90640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.902+0000 7f907cff9700 1 -- 172.21.15.7:0/1498239053 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9074083a20 con 0x7f90800cb0c0 2023-10-07T07:57:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.902+0000 7f907ffff700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9064066f30 0x7f90640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.903+0000 7f907ffff700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9064066f30 0x7f90640693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f906c0093f0 tx=0x7f906c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:50.906+0000 7f907cff9700 1 -- 172.21.15.7:0/1498239053 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9074054720 con 0x7f90800cb0c0 2023-10-07T07:57:51.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.107+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f908016a2e0 con 0x7f90800cb0c0 2023-10-07T07:57:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.108+0000 7f907cff9700 1 -- 172.21.15.7:0/1498239053 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f9074017020 con 0x7f90800cb0c0 2023-10-07T07:57:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:51.110 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:50.139000+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9064066f30 msgr2=0x7f90640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9064066f30 0x7f90640693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f906c0093f0 tx=0x7f906c017000 comp rx=0 tx=0).stop 2023-10-07T07:57:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90800cb0c0 msgr2=0x7f90800ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90800cb0c0 0x7f90800ae430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f907400c7c0 tx=0x7f9074007ea0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 shutdown_connections 2023-10-07T07:57:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9064066f30 0x7f90640693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9080159690 0x7f908015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90800cb0c0 0x7f90800ae430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 --2- 172.21.15.7:0/1498239053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90801592c0 0x7f90801694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 >> 172.21.15.7:0/1498239053 conn(0x7f9080068bd0 msgr2=0x7f90800c1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 shutdown_connections 2023-10-07T07:57:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.110+0000 7f9086c9f700 1 -- 172.21.15.7:0/1498239053 wait complete. 2023-10-07T07:57:51.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.258+0000 7f8bd9a0b700 1 Processor -- start 2023-10-07T07:57:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.258+0000 7f8bd9a0b700 1 -- start start 2023-10-07T07:57:51.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.259+0000 7f8bd9a0b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8bd4159380 0x7f8bd40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.259+0000 7f8bd9a0b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 0x7f8bd40d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.259+0000 7f8bd9a0b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd40d0be0 con 0x7f8bd40b3b60 2023-10-07T07:57:51.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.259+0000 7f8bd9a0b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd40b4820 con 0x7f8bd4159380 2023-10-07T07:57:51.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.259+0000 7f8bd9a0b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40b5000 con 0x7f8bd40d0250 2023-10-07T07:57:51.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.259+0000 7f8bd2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8bd4159380 0x7f8bd40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.259+0000 7f8bd2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8bd4159380 0x7f8bd40b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33406/0 (socket says 172.21.15.7:33406) 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.259+0000 7f8bd2ffd700 1 -- 172.21.15.7:0/1005247112 learned_addr learned my addr 172.21.15.7:0/1005247112 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.260+0000 7f8bcbfff700 1 --2- 172.21.15.7:0/1005247112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 0x7f8bd40d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.260+0000 7f8bd2ffd700 1 -- 172.21.15.7:0/1005247112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 msgr2=0x7f8bd40d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.260+0000 7f8bd2ffd700 1 --2- 172.21.15.7:0/1005247112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 0x7f8bd40d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.260+0000 7f8bd2ffd700 1 -- 172.21.15.7:0/1005247112 >> v1:172.21.15.172:6789/0 conn(0x7f8bd40b3b60 legacy=0x7f8bd40b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.260+0000 7f8bd2ffd700 1 -- 172.21.15.7:0/1005247112 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd40b52e0 con 0x7f8bd4159380 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.260+0000 7f8bd2ffd700 1 --2- 172.21.15.7:0/1005247112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd4159380 0x7f8bd40b5ae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40099b0 tx=0x7f8bc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6164c63cb893ef22 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.260+0000 7f8bd27fc700 1 -- 172.21.15.7:0/1005247112 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400b860 con 0x7f8bd4159380 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.260+0000 7f8bd27fc700 1 -- 172.21.15.7:0/1005247112 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc400b9c0 con 0x7f8bd4159380 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.260+0000 7f8bd27fc700 1 -- 172.21.15.7:0/1005247112 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400bd60 con 0x7f8bd4159380 2023-10-07T07:57:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.261+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/1005247112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd4159380 msgr2=0x7f8bd40b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.261+0000 7f8bd9a0b700 1 --2- 172.21.15.7:0/1005247112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd4159380 0x7f8bd40b5ae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40099b0 tx=0x7f8bc4009250 comp rx=0 tx=0).stop 2023-10-07T07:57:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.261+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/1005247112 shutdown_connections 2023-10-07T07:57:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.261+0000 7f8bd9a0b700 1 --2- 172.21.15.7:0/1005247112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd4159380 0x7f8bd40b5ae0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.261+0000 7f8bd9a0b700 1 --2- 172.21.15.7:0/1005247112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 0x7f8bd40d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.261+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/1005247112 >> 172.21.15.7:0/1005247112 conn(0x7f8bd4067be0 msgr2=0x7f8bd4067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.261+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/1005247112 shutdown_connections 2023-10-07T07:57:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.261+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/1005247112 wait complete. 2023-10-07T07:57:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.262+0000 7f8bd9a0b700 1 Processor -- start 2023-10-07T07:57:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.262+0000 7f8bd9a0b700 1 -- start start 2023-10-07T07:57:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd9a0b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40b3b60 0x7f8bd4068260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd9a0b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 0x7f8bd415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd9a0b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd4159380 0x7f8bd415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd9a0b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd415c1b0 con 0x7f8bd40b3b60 2023-10-07T07:57:51.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd9a0b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd4169bf0 con 0x7f8bd4159380 2023-10-07T07:57:51.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd9a0b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd415c3b0 con 0x7f8bd40d0250 2023-10-07T07:57:51.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40b3b60 0x7f8bd4068260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40b3b60 0x7f8bd4068260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33418/0 (socket says 172.21.15.7:33418) 2023-10-07T07:57:51.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd2ffd700 1 -- 172.21.15.7:0/3393449462 learned_addr learned my addr 172.21.15.7:0/3393449462 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:51.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bcbfff700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 0x7f8bd415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd37fe700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd4159380 0x7f8bd415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd2ffd700 1 -- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 msgr2=0x7f8bd415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd2ffd700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 0x7f8bd415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd2ffd700 1 -- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd4159380 msgr2=0x7f8bd415bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd2ffd700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd4159380 0x7f8bd415bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd2ffd700 1 -- 172.21.15.7:0/3393449462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40094f0 con 0x7f8bd40b3b60 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.263+0000 7f8bd2ffd700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40b3b60 0x7f8bd4068260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8bd406f850 tx=0x7f8bc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.264+0000 7f8bcaffd700 1 -- 172.21.15.7:0/3393449462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4011e40 con 0x7f8bd40b3b60 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.264+0000 7f8bcaffd700 1 -- 172.21.15.7:0/3393449462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc4024440 con 0x7f8bd40b3b60 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.264+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd4169eb0 con 0x7f8bd40b3b60 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.264+0000 7f8bcaffd700 1 -- 172.21.15.7:0/3393449462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4024800 con 0x7f8bd40b3b60 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.264+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd416a470 con 0x7f8bd40b3b60 2023-10-07T07:57:51.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.265+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd40716c0 con 0x7f8bd40b3b60 2023-10-07T07:57:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.265+0000 7f8bcaffd700 1 -- 172.21.15.7:0/3393449462 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8bc402d440 con 0x7f8bd40b3b60 2023-10-07T07:57:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.266+0000 7f8bcaffd700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8bb4066f30 0x7f8bb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.266+0000 7f8bcaffd700 1 -- 172.21.15.7:0/3393449462 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8bc4085e10 con 0x7f8bd40b3b60 2023-10-07T07:57:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.266+0000 7f8bcbfff700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8bb4066f30 0x7f8bb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.266+0000 7f8bcbfff700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8bb4066f30 0x7f8bb40693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006fd0 tx=0x7f8bbc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.270+0000 7f8bcaffd700 1 -- 172.21.15.7:0/3393449462 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bc4053080 con 0x7f8bd40b3b60 2023-10-07T07:57:51.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.475+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bd40cd120 con 0x7f8bd40b3b60 2023-10-07T07:57:51.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.476+0000 7f8bcaffd700 1 -- 172.21.15.7:0/3393449462 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f8bc40566a0 con 0x7f8bd40b3b60 2023-10-07T07:57:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:51.477 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:50.139000+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8bb4066f30 msgr2=0x7f8bb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8bb4066f30 0x7f8bb40693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006fd0 tx=0x7f8bbc008040 comp rx=0 tx=0).stop 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40b3b60 msgr2=0x7f8bd4068260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40b3b60 0x7f8bd4068260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8bd406f850 tx=0x7f8bc4017000 comp rx=0 tx=0).stop 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 shutdown_connections 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8bb4066f30 0x7f8bb40693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd4159380 0x7f8bd415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40b3b60 0x7f8bd4068260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 --2- 172.21.15.7:0/3393449462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd40d0250 0x7f8bd415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 >> 172.21.15.7:0/3393449462 conn(0x7f8bd4067be0 msgr2=0x7f8bd40afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 shutdown_connections 2023-10-07T07:57:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.478+0000 7f8bd9a0b700 1 -- 172.21.15.7:0/3393449462 wait complete. 2023-10-07T07:57:51.493 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-07T07:57:51.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.627+0000 7ff49eeb0700 1 Processor -- start 2023-10-07T07:57:51.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.627+0000 7ff49eeb0700 1 -- start start 2023-10-07T07:57:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.628+0000 7ff49eeb0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980b39b0 0x7ff498068100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.628+0000 7ff49eeb0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff498068640 con 0x7ff4980b14a0 2023-10-07T07:57:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.628+0000 7ff49eeb0700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980b1130 con 0x7ff4980b0190 2023-10-07T07:57:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.628+0000 7ff49eeb0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49815a8e0 con 0x7ff4980b39b0 2023-10-07T07:57:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.628+0000 7ff497fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff4980b14a0 0x7ff4980afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35906/0 (socket says 172.21.15.7:35906) 2023-10-07T07:57:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.628+0000 7ff497fff700 1 -- 172.21.15.7:0/4097801424 learned_addr learned my addr 172.21.15.7:0/4097801424 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.628+0000 7ff49cc4c700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980b39b0 0x7ff498068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.628+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879650848 0 0) 0x7ff498068640 con 0x7ff4980b14a0 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.628+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff480003570 con 0x7ff4980b14a0 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107261337 0 0) 0x7ff4980b1130 con 0x7ff4980b0190 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff498068640 con 0x7ff4980b0190 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038258104 0 0) 0x7ff480003570 con 0x7ff4980b14a0 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980b39b0 msgr2=0x7ff498068100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980b39b0 0x7ff498068100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 >> v1:172.21.15.172:6789/0 conn(0x7ff4980b0190 legacy=0x7ff49815d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498159cd0 con 0x7ff4980b14a0 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7ff48c001470 con 0x7ff4980b14a0 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 >> v1:172.21.15.7:6789/0 conn(0x7ff4980b14a0 legacy=0x7ff4980afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980b39b0 0x7ff480003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4800041d0 0x7ff480004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff480004d50 0x7ff4800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49815a8e0 con 0x7ff480004d50 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48000bbf0 con 0x7ff4800041d0 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48000bdb0 con 0x7ff4980b39b0 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff49d44d700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff480004d50 0x7ff4800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff49cc4c700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980b39b0 0x7ff480003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff497fff700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4800041d0 0x7ff480004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff49d44d700 1 -- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980b39b0 msgr2=0x7ff480003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff49d44d700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980b39b0 0x7ff480003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff49d44d700 1 -- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4800041d0 msgr2=0x7ff480004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff49d44d700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4800041d0 0x7ff480004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff49d44d700 1 -- 172.21.15.7:0/4097801424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480003570 con 0x7ff480004d50 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.629+0000 7ff49d44d700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff480004d50 0x7ff4800051c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff490002120 tx=0x7ff4900017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffcfee29591f436 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490001ab0 con 0x7ff480004d50 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49000c980 con 0x7ff480004d50 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff4977fe700 1 -- 172.21.15.7:0/4097801424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49000cd20 con 0x7ff480004d50 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff49eeb0700 1 -- 172.21.15.7:0/4097801424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff480004d50 msgr2=0x7ff4800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff49eeb0700 1 --2- 172.21.15.7:0/4097801424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff480004d50 0x7ff4800051c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff490002120 tx=0x7ff4900017a0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff49d44d700 1 -- 172.21.15.7:0/4097801424 reap_dead start 2023-10-07T07:57:51.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff49eeb0700 1 -- 172.21.15.7:0/4097801424 shutdown_connections 2023-10-07T07:57:51.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff49eeb0700 1 -- 172.21.15.7:0/4097801424 >> 172.21.15.7:0/4097801424 conn(0x7ff49806d350 msgr2=0x7ff49806d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:51.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff49eeb0700 1 -- 172.21.15.7:0/4097801424 shutdown_connections 2023-10-07T07:57:51.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.630+0000 7ff49eeb0700 1 -- 172.21.15.7:0/4097801424 wait complete. 2023-10-07T07:57:51.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.632+0000 7ff49eeb0700 1 Processor -- start 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.632+0000 7ff49eeb0700 1 -- start start 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49eeb0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff49814ba80 0x7ff4980c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49eeb0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814be50 0x7ff4980cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49eeb0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980c83e0 0x7ff4980ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49eeb0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980a5480 con 0x7ff49814be50 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49eeb0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980c97f0 con 0x7ff49814ba80 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49eeb0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980a56d0 con 0x7ff4980c83e0 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff497fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814be50 0x7ff4980cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff497fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814be50 0x7ff4980cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33446/0 (socket says 172.21.15.7:33446) 2023-10-07T07:57:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff497fff700 1 -- 172.21.15.7:0/3782954208 learned_addr learned my addr 172.21.15.7:0/3782954208 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49cc4c700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff49814ba80 0x7ff4980c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49d44d700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980c83e0 0x7ff4980ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff497fff700 1 -- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980c83e0 msgr2=0x7ff4980ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff497fff700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980c83e0 0x7ff4980ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff497fff700 1 -- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff49814ba80 msgr2=0x7ff4980c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff497fff700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff49814ba80 0x7ff4980c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff497fff700 1 -- 172.21.15.7:0/3782954208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980a59b0 con 0x7ff49814be50 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff497fff700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814be50 0x7ff4980cca80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0070e0 tx=0x7ff48c001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff494ff9700 1 -- 172.21.15.7:0/3782954208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c018030 con 0x7ff49814be50 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff494ff9700 1 -- 172.21.15.7:0/3782954208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c00f610 con 0x7ff49814be50 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4980a5c90 con 0x7ff49814be50 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff494ff9700 1 -- 172.21.15.7:0/3782954208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00f9b0 con 0x7ff49814be50 2023-10-07T07:57:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.633+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff498169670 con 0x7ff49814be50 2023-10-07T07:57:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.635+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4980a61b0 con 0x7ff49814be50 2023-10-07T07:57:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.635+0000 7ff494ff9700 1 -- 172.21.15.7:0/3782954208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff48c01b070 con 0x7ff49814be50 2023-10-07T07:57:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.635+0000 7ff494ff9700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff484066f30 0x7ff4840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.635+0000 7ff494ff9700 1 -- 172.21.15.7:0/3782954208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff48c019070 con 0x7ff49814be50 2023-10-07T07:57:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.636+0000 7ff49cc4c700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff484066f30 0x7ff4840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.636+0000 7ff49cc4c700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff484066f30 0x7ff4840693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff488006f00 tx=0x7ff488008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.640+0000 7ff494ff9700 1 -- 172.21.15.7:0/3782954208 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4980a61b0 con 0x7ff49814be50 2023-10-07T07:57:51.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:51.890+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff4980a61b0 con 0x7ff49814be50 2023-10-07T07:57:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.727+0000 7ff494ff9700 1 -- 172.21.15.7:0/3782954208 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff4980a61b0 con 0x7ff49814be50 2023-10-07T07:57:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff484066f30 msgr2=0x7ff4840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff484066f30 0x7ff4840693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff488006f00 tx=0x7ff488008040 comp rx=0 tx=0).stop 2023-10-07T07:57:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814be50 msgr2=0x7ff4980cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814be50 0x7ff4980cca80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0070e0 tx=0x7ff48c001d20 comp rx=0 tx=0).stop 2023-10-07T07:57:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 shutdown_connections 2023-10-07T07:57:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff484066f30 0x7ff4840693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4980c83e0 0x7ff4980ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff49814ba80 0x7ff4980c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 --2- 172.21.15.7:0/3782954208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff49814be50 0x7ff4980cca80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.730+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 >> 172.21.15.7:0/3782954208 conn(0x7ff49806d350 msgr2=0x7ff4980af730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.731+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 shutdown_connections 2023-10-07T07:57:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.731+0000 7ff49eeb0700 1 -- 172.21.15.7:0/3782954208 wait complete. 2023-10-07T07:57:52.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-07T07:57:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.884+0000 7fe771f91700 1 Processor -- start 2023-10-07T07:57:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.884+0000 7fe771f91700 1 -- start start 2023-10-07T07:57:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe771f91700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c0c8800 0x7fe76c13f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe771f91700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76c0c9510 con 0x7fe76c06e5e0 2023-10-07T07:57:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe771f91700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76c0c8c70 con 0x7fe76c06e9b0 2023-10-07T07:57:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe771f91700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c0c8f50 con 0x7fe76c0c8800 2023-10-07T07:57:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe76c06e5e0 0x7fe76c0c80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35920/0 (socket says 172.21.15.7:35920) 2023-10-07T07:57:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76b7fe700 1 -- 172.21.15.7:0/3515771496 learned_addr learned my addr 172.21.15.7:0/3515771496 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76affd700 1 --2- 172.21.15.7:0/3515771496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c0c8800 0x7fe76c13f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76a7fc700 1 -- 172.21.15.7:0/3515771496 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923005410 0 0) 0x7fe76c0c9510 con 0x7fe76c06e5e0 2023-10-07T07:57:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76a7fc700 1 -- 172.21.15.7:0/3515771496 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe758003570 con 0x7fe76c06e5e0 2023-10-07T07:57:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76a7fc700 1 -- 172.21.15.7:0/3515771496 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857893876 0 0) 0x7fe76c0c8c70 con 0x7fe76c06e9b0 2023-10-07T07:57:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76a7fc700 1 -- 172.21.15.7:0/3515771496 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe76c0c9510 con 0x7fe76c06e9b0 2023-10-07T07:57:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76affd700 1 -- 172.21.15.7:0/3515771496 >> v1:172.21.15.172:6789/0 conn(0x7fe76c06e9b0 legacy=0x7fe76c13f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76affd700 1 -- 172.21.15.7:0/3515771496 >> v1:172.21.15.7:6789/0 conn(0x7fe76c06e5e0 legacy=0x7fe76c0c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.885+0000 7fe76affd700 1 -- 172.21.15.7:0/3515771496 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c0c9230 con 0x7fe76c0c8800 2023-10-07T07:57:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.886+0000 7fe76affd700 1 --2- 172.21.15.7:0/3515771496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c0c8800 0x7fe76c13f350 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009910 tx=0x7fe75c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=285cba269e102f54 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.886+0000 7fe76a7fc700 1 -- 172.21.15.7:0/3515771496 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c011630 con 0x7fe76c0c8800 2023-10-07T07:57:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.886+0000 7fe76a7fc700 1 -- 172.21.15.7:0/3515771496 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c011790 con 0x7fe76c0c8800 2023-10-07T07:57:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.886+0000 7fe76a7fc700 1 -- 172.21.15.7:0/3515771496 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c011b30 con 0x7fe76c0c8800 2023-10-07T07:57:52.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.886+0000 7fe771f91700 1 -- 172.21.15.7:0/3515771496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c0c8800 msgr2=0x7fe76c13f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:52.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.886+0000 7fe771f91700 1 --2- 172.21.15.7:0/3515771496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c0c8800 0x7fe76c13f350 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009910 tx=0x7fe75c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:52.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.886+0000 7fe771f91700 1 -- 172.21.15.7:0/3515771496 shutdown_connections 2023-10-07T07:57:52.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.886+0000 7fe771f91700 1 --2- 172.21.15.7:0/3515771496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c0c8800 0x7fe76c13f350 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.886+0000 7fe771f91700 1 -- 172.21.15.7:0/3515771496 >> 172.21.15.7:0/3515771496 conn(0x7fe76c0ccf30 msgr2=0x7fe76c0cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.887+0000 7fe771f91700 1 -- 172.21.15.7:0/3515771496 shutdown_connections 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.887+0000 7fe771f91700 1 -- 172.21.15.7:0/3515771496 wait complete. 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.888+0000 7fe771f91700 1 Processor -- start 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.888+0000 7fe771f91700 1 -- start start 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe771f91700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c06e5e0 0x7fe76c16a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe771f91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06e9b0 0x7fe76c156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe771f91700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe76c0c8800 0x7fe76c13c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe771f91700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c156f40 con 0x7fe76c06e9b0 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe771f91700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c15a9d0 con 0x7fe76c0c8800 2023-10-07T07:57:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe771f91700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c15b550 con 0x7fe76c06e5e0 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06e9b0 0x7fe76c156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06e9b0 0x7fe76c156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33462/0 (socket says 172.21.15.7:33462) 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76affd700 1 -- 172.21.15.7:0/3195419880 learned_addr learned my addr 172.21.15.7:0/3195419880 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76b7fe700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c06e5e0 0x7fe76c16a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76bfff700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe76c0c8800 0x7fe76c13c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76affd700 1 -- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c06e5e0 msgr2=0x7fe76c16a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76affd700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c06e5e0 0x7fe76c16a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76affd700 1 -- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe76c0c8800 msgr2=0x7fe76c13c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76affd700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe76c0c8800 0x7fe76c13c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76affd700 1 -- 172.21.15.7:0/3195419880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c009420 con 0x7fe76c06e9b0 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe76affd700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06e9b0 0x7fe76c156a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009fd0 tx=0x7fe75c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.889+0000 7fe7637fe700 1 -- 172.21.15.7:0/3195419880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c01f070 con 0x7fe76c06e9b0 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.890+0000 7fe7637fe700 1 -- 172.21.15.7:0/3195419880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c002f70 con 0x7fe76c06e9b0 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.890+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76c15ac20 con 0x7fe76c06e9b0 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.890+0000 7fe7637fe700 1 -- 172.21.15.7:0/3195419880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c011900 con 0x7fe76c06e9b0 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.890+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c15b1e0 con 0x7fe76c06e9b0 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.891+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76c0716c0 con 0x7fe76c06e9b0 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.891+0000 7fe7637fe700 1 -- 172.21.15.7:0/3195419880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe75c023070 con 0x7fe76c06e9b0 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.891+0000 7fe7637fe700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe74c066f30 0x7fe74c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.892+0000 7fe7637fe700 1 -- 172.21.15.7:0/3195419880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe75c0856b0 con 0x7fe76c06e9b0 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.892+0000 7fe76b7fe700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe74c066f30 0x7fe74c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.892+0000 7fe76b7fe700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe74c066f30 0x7fe74c0693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe754007fd0 tx=0x7fe754009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:52.896+0000 7fe7637fe700 1 -- 172.21.15.7:0/3195419880 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe75c052920 con 0x7fe76c06e9b0 2023-10-07T07:57:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.148+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe76c15b690 con 0x7fe76c06e9b0 2023-10-07T07:57:53.628 INFO:tasks.ceph.mds.c.smithi007.stderr:2023-10-07T07:57:53.625+0000 7fba013fd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T07:57:53.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.728+0000 7fe7637fe700 1 -- 172.21.15.7:0/3195419880 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe75c0150c0 con 0x7fe76c06e9b0 2023-10-07T07:57:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe74c066f30 msgr2=0x7fe74c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe74c066f30 0x7fe74c0693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe754007fd0 tx=0x7fe754009040 comp rx=0 tx=0).stop 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06e9b0 msgr2=0x7fe76c156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06e9b0 0x7fe76c156a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009fd0 tx=0x7fe75c017000 comp rx=0 tx=0).stop 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 shutdown_connections 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe74c066f30 0x7fe74c0693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe76c0c8800 0x7fe76c13c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76c06e5e0 0x7fe76c16a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 --2- 172.21.15.7:0/3195419880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06e9b0 0x7fe76c156a00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 >> 172.21.15.7:0/3195419880 conn(0x7fe76c0ccf30 msgr2=0x7fe76c0ca230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 shutdown_connections 2023-10-07T07:57:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.731+0000 7fe771f91700 1 -- 172.21.15.7:0/3195419880 wait complete. 2023-10-07T07:57:53.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:57:53.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.881+0000 7fcd0d11e700 1 Processor -- start 2023-10-07T07:57:53.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.881+0000 7fcd0d11e700 1 -- start start 2023-10-07T07:57:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.882+0000 7fcd0d11e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd081592c0 0x7fcd0806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.882+0000 7fcd0d11e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd080cb290 0x7fcd0806a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.882+0000 7fcd0d11e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0806bb70 con 0x7fcd08159690 2023-10-07T07:57:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.882+0000 7fcd0d11e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd080cda10 con 0x7fcd080cb290 2023-10-07T07:57:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.882+0000 7fcd0d11e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0806af60 con 0x7fcd081592c0 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.882+0000 7fcd0659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcd08159690 0x7fcd080cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35926/0 (socket says 172.21.15.7:35926) 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.882+0000 7fcd0659c700 1 -- 172.21.15.7:0/3564355391 learned_addr learned my addr 172.21.15.7:0/3564355391 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.882+0000 7fcd0759e700 1 --2- 172.21.15.7:0/3564355391 >> v2:172.21.15.7:3300/0 conn(0x7fcd080cb290 0x7fcd0806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.882+0000 7fcd06d9d700 1 --2- 172.21.15.7:0/3564355391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd081592c0 0x7fcd0806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.883+0000 7fcd0759e700 1 -- 172.21.15.7:0/3564355391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd081592c0 msgr2=0x7fcd0806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.883+0000 7fcd0759e700 1 --2- 172.21.15.7:0/3564355391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd081592c0 0x7fcd0806a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.883+0000 7fcd0759e700 1 -- 172.21.15.7:0/3564355391 >> v1:172.21.15.7:6789/0 conn(0x7fcd08159690 legacy=0x7fcd080cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.883+0000 7fcd05d9b700 1 -- 172.21.15.7:0/3564355391 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739002401 0 0) 0x7fcd0806bb70 con 0x7fcd08159690 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.883+0000 7fcd0759e700 1 -- 172.21.15.7:0/3564355391 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0806b240 con 0x7fcd080cb290 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.883+0000 7fcd0759e700 1 --2- 172.21.15.7:0/3564355391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd080cb290 0x7fcd0806a930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009e60 tx=0x7fcd00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8f3857cb3d1973 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.883+0000 7fcd05d9b700 1 -- 172.21.15.7:0/3564355391 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000e070 con 0x7fcd080cb290 2023-10-07T07:57:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.883+0000 7fcd05d9b700 1 -- 172.21.15.7:0/3564355391 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0000bb60 con 0x7fcd080cb290 2023-10-07T07:57:53.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.883+0000 7fcd05d9b700 1 -- 172.21.15.7:0/3564355391 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00012430 con 0x7fcd080cb290 2023-10-07T07:57:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.884+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3564355391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd080cb290 msgr2=0x7fcd0806a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.884+0000 7fcd0d11e700 1 --2- 172.21.15.7:0/3564355391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd080cb290 0x7fcd0806a930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009e60 tx=0x7fcd00009780 comp rx=0 tx=0).stop 2023-10-07T07:57:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.884+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3564355391 shutdown_connections 2023-10-07T07:57:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.884+0000 7fcd0d11e700 1 --2- 172.21.15.7:0/3564355391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd080cb290 0x7fcd0806a930 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.884+0000 7fcd0d11e700 1 --2- 172.21.15.7:0/3564355391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd081592c0 0x7fcd0806a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.884+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3564355391 >> 172.21.15.7:0/3564355391 conn(0x7fcd08068bd0 msgr2=0x7fcd080ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.884+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3564355391 shutdown_connections 2023-10-07T07:57:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.884+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3564355391 wait complete. 2023-10-07T07:57:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.886+0000 7fcd0d11e700 1 Processor -- start 2023-10-07T07:57:53.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.886+0000 7fcd0d11e700 1 -- start start 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0d11e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd080cb290 0x7fcd08143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0d11e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd081592c0 0x7fcd080ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0d11e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd08159690 0x7fcd080c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0d11e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080ae940 con 0x7fcd08159690 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0d11e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080aedd0 con 0x7fcd081592c0 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0d11e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080add30 con 0x7fcd080cb290 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd08159690 0x7fcd080c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd08159690 0x7fcd080c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33486/0 (socket says 172.21.15.7:33486) 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0759e700 1 -- 172.21.15.7:0/3413030258 learned_addr learned my addr 172.21.15.7:0/3413030258 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd06d9d700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd080cb290 0x7fcd08143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0659c700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd081592c0 0x7fcd080ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0759e700 1 -- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd080cb290 msgr2=0x7fcd08143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0759e700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd080cb290 0x7fcd08143580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0759e700 1 -- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd081592c0 msgr2=0x7fcd080ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0759e700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd081592c0 0x7fcd080ad3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0759e700 1 -- 172.21.15.7:0/3413030258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00009a20 con 0x7fcd08159690 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0759e700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd08159690 0x7fcd080c3250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000b6c0 tx=0x7fcd00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fccfeffd700 1 -- 172.21.15.7:0/3413030258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00027470 con 0x7fcd08159690 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fccfeffd700 1 -- 172.21.15.7:0/3413030258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd000275d0 con 0x7fcd08159690 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd080ae010 con 0x7fcd08159690 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.887+0000 7fccfeffd700 1 -- 172.21.15.7:0/3413030258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00027990 con 0x7fcd08159690 2023-10-07T07:57:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.888+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd080ae5d0 con 0x7fcd08159690 2023-10-07T07:57:53.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.888+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd080aeab0 con 0x7fcd08159690 2023-10-07T07:57:53.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.889+0000 7fccfeffd700 1 -- 172.21.15.7:0/3413030258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd00024070 con 0x7fcd08159690 2023-10-07T07:57:53.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.889+0000 7fccfeffd700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcce8066f30 0x7fcce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:53.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.889+0000 7fccfeffd700 1 -- 172.21.15.7:0/3413030258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd00086f10 con 0x7fcd08159690 2023-10-07T07:57:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.889+0000 7fcd06d9d700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcce8066f30 0x7fcce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.890+0000 7fcd06d9d700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcce8066f30 0x7fcce80693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009d40 tx=0x7fccf0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:53.894+0000 7fccfeffd700 1 -- 172.21.15.7:0/3413030258 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd00054180 con 0x7fcd08159690 2023-10-07T07:57:54.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.140+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd080716c0 con 0x7fcd08159690 2023-10-07T07:57:54.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.140+0000 7fccfeffd700 1 -- 172.21.15.7:0/3413030258 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fcd000577a0 con 0x7fcd08159690 2023-10-07T07:57:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:54.142 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2318752844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2318752844},{"type":"v1","addr":"172.21.15.172:6836","nonce":2318752844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1231430544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1231430544},{"type":"v1","addr":"172.21.15.172:6837","nonce":1231430544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1421504798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1421504798},{"type":"v1","addr":"172.21.15.7:6833","nonce":1421504798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:57:50.139330+0000","modified":"2023-10-07T07:57:53.727131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T07:57:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcce8066f30 msgr2=0x7fcce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcce8066f30 0x7fcce80693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009d40 tx=0x7fccf0008040 comp rx=0 tx=0).stop 2023-10-07T07:57:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd08159690 msgr2=0x7fcd080c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd08159690 0x7fcd080c3250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000b6c0 tx=0x7fcd00019000 comp rx=0 tx=0).stop 2023-10-07T07:57:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 shutdown_connections 2023-10-07T07:57:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcce8066f30 0x7fcce80693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd08159690 0x7fcd080c3250 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd080cb290 0x7fcd08143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 --2- 172.21.15.7:0/3413030258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd081592c0 0x7fcd080ad3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 >> 172.21.15.7:0/3413030258 conn(0x7fcd08068bd0 msgr2=0x7fcd0806e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 shutdown_connections 2023-10-07T07:57:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.143+0000 7fcd0d11e700 1 -- 172.21.15.7:0/3413030258 wait complete. 2023-10-07T07:57:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2023-10-07T07:57:54.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.288+0000 7f9a44761700 1 Processor -- start 2023-10-07T07:57:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.288+0000 7f9a44761700 1 -- start start 2023-10-07T07:57:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.289+0000 7f9a44761700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a3c06e9d0 0x7f9a3c0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.289+0000 7f9a44761700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3c06b700 con 0x7f9a3c155f20 2023-10-07T07:57:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.289+0000 7f9a44761700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3c06aaf0 con 0x7f9a3c1562f0 2023-10-07T07:57:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.289+0000 7f9a44761700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3c06add0 con 0x7f9a3c06e9d0 2023-10-07T07:57:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.289+0000 7f9a424fd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9a3c155f20 0x7f9a3c06e2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35928/0 (socket says 172.21.15.7:35928) 2023-10-07T07:57:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.289+0000 7f9a424fd700 1 -- 172.21.15.7:0/161147026 learned_addr learned my addr 172.21.15.7:0/161147026 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a41cfc700 1 --2- 172.21.15.7:0/161147026 >> v2:172.21.15.172:3300/0 conn(0x7f9a3c06e9d0 0x7f9a3c0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189134695 0 0) 0x7f9a3c06b700 con 0x7f9a3c155f20 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a28003570 con 0x7f9a3c155f20 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034958995 0 0) 0x7f9a28003570 con 0x7f9a3c155f20 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 >> v2:172.21.15.172:3300/0 conn(0x7f9a3c06e9d0 msgr2=0x7f9a3c0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 --2- 172.21.15.7:0/161147026 >> v2:172.21.15.172:3300/0 conn(0x7f9a3c06e9d0 0x7f9a3c0c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 >> v1:172.21.15.172:6789/0 conn(0x7f9a3c1562f0 legacy=0x7f9a3c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c06b0b0 con 0x7f9a3c155f20 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f9a30001470 con 0x7f9a3c155f20 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 >> v1:172.21.15.7:6789/0 conn(0x7f9a3c155f20 legacy=0x7f9a3c06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a28003a60 0x7f9a28003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a28004810 0x7f9a28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a2800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c06add0 con 0x7f9a280051c0 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2800c0d0 con 0x7f9a28003a60 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2800c290 con 0x7f9a28004810 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a42cfe700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a2800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a424fd700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a28003a60 0x7f9a28003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.290+0000 7f9a41cfc700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a28004810 0x7f9a28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a42cfe700 1 -- 172.21.15.7:0/161147026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a28004810 msgr2=0x7f9a28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a42cfe700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a28004810 0x7f9a28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a42cfe700 1 -- 172.21.15.7:0/161147026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a28003a60 msgr2=0x7f9a28003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a42cfe700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a28003a60 0x7f9a28003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a42cfe700 1 -- 172.21.15.7:0/161147026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a28003570 con 0x7f9a280051c0 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a42cfe700 1 -- 172.21.15.7:0/161147026 reap_dead start 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a42cfe700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a2800ba60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a34001ff0 tx=0x7f9a34001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5184146926a0fb90 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34001980 con 0x7f9a280051c0 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3400c8b0 con 0x7f9a280051c0 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a414fb700 1 -- 172.21.15.7:0/161147026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400cc50 con 0x7f9a280051c0 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a44761700 1 -- 172.21.15.7:0/161147026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 msgr2=0x7f9a2800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.291+0000 7f9a44761700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a2800ba60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a34001ff0 tx=0x7f9a34001670 comp rx=0 tx=0).stop 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.292+0000 7f9a44761700 1 -- 172.21.15.7:0/161147026 shutdown_connections 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.292+0000 7f9a44761700 1 --2- 172.21.15.7:0/161147026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a2800ba60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.292+0000 7f9a44761700 1 -- 172.21.15.7:0/161147026 >> 172.21.15.7:0/161147026 conn(0x7f9a3c068bd0 msgr2=0x7f9a3c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.292+0000 7f9a44761700 1 -- 172.21.15.7:0/161147026 shutdown_connections 2023-10-07T07:57:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.292+0000 7f9a44761700 1 -- 172.21.15.7:0/161147026 wait complete. 2023-10-07T07:57:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.293+0000 7f9a44761700 1 Processor -- start 2023-10-07T07:57:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.293+0000 7f9a44761700 1 -- start start 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.293+0000 7f9a44761700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a3c0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.293+0000 7f9a44761700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c0c6820 0x7f9a3c066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.293+0000 7f9a44761700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0c6c90 0x7f9a3c067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a44761700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0b2d20 con 0x7f9a280051c0 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a44761700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0c7230 con 0x7f9a3c0c6820 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a44761700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0c7510 con 0x7f9a3c0c6c90 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a424fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a3c0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a424fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a3c0b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33504/0 (socket says 172.21.15.7:33504) 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a424fd700 1 -- 172.21.15.7:0/1505500930 learned_addr learned my addr 172.21.15.7:0/1505500930 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a42cfe700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0c6c90 0x7f9a3c067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a41cfc700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c0c6820 0x7f9a3c066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a424fd700 1 -- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0c6c90 msgr2=0x7f9a3c067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a424fd700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0c6c90 0x7f9a3c067530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a424fd700 1 -- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c0c6820 msgr2=0x7f9a3c066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a424fd700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c0c6820 0x7f9a3c066ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a424fd700 1 -- 172.21.15.7:0/1505500930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c0b02a0 con 0x7f9a280051c0 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a424fd700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a3c0b27e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a30007390 tx=0x7f9a30001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a2e7fc700 1 -- 172.21.15.7:0/1505500930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30018030 con 0x7f9a280051c0 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a2e7fc700 1 -- 172.21.15.7:0/1505500930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3000f8d0 con 0x7f9a280051c0 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c0b0580 con 0x7f9a280051c0 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a2e7fc700 1 -- 172.21.15.7:0/1505500930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3000fc90 con 0x7f9a280051c0 2023-10-07T07:57:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.294+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c0c4180 con 0x7f9a280051c0 2023-10-07T07:57:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.295+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c0c43a0 con 0x7f9a280051c0 2023-10-07T07:57:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.296+0000 7f9a2e7fc700 1 -- 172.21.15.7:0/1505500930 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a3001b070 con 0x7f9a280051c0 2023-10-07T07:57:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.296+0000 7f9a2e7fc700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9a20066f30 0x7f9a200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.296+0000 7f9a2e7fc700 1 -- 172.21.15.7:0/1505500930 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a30019070 con 0x7f9a280051c0 2023-10-07T07:57:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.296+0000 7f9a41cfc700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9a20066f30 0x7f9a200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.297+0000 7f9a41cfc700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9a20066f30 0x7f9a200693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c154ee0 tx=0x7f9a38008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.301+0000 7f9a2e7fc700 1 -- 172.21.15.7:0/1505500930 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a3c0c43a0 con 0x7f9a280051c0 2023-10-07T07:57:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.503+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a3c0c43a0 con 0x7f9a280051c0 2023-10-07T07:57:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.504+0000 7f9a2e7fc700 1 -- 172.21.15.7:0/1505500930 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f9a3c0c43a0 con 0x7f9a280051c0 2023-10-07T07:57:54.505 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:54.506 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.506+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9a20066f30 msgr2=0x7f9a200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.506+0000 7f9a44761700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9a20066f30 0x7f9a200693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c154ee0 tx=0x7f9a38008040 comp rx=0 tx=0).stop 2023-10-07T07:57:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.506+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 msgr2=0x7f9a3c0b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.507+0000 7f9a44761700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a3c0b27e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a30007390 tx=0x7f9a30001e50 comp rx=0 tx=0).stop 2023-10-07T07:57:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.507+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 shutdown_connections 2023-10-07T07:57:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.507+0000 7f9a44761700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9a20066f30 0x7f9a200693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.507+0000 7f9a44761700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a3c0c6c90 0x7f9a3c067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.507+0000 7f9a44761700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280051c0 0x7f9a3c0b27e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.507+0000 7f9a44761700 1 --2- 172.21.15.7:0/1505500930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a3c0c6820 0x7f9a3c066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.507+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 >> 172.21.15.7:0/1505500930 conn(0x7f9a3c068bd0 msgr2=0x7f9a3c155060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:54.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.507+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 shutdown_connections 2023-10-07T07:57:54.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.507+0000 7f9a44761700 1 -- 172.21.15.7:0/1505500930 wait complete. 2023-10-07T07:57:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-07T07:57:54.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T07:57:54.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.652+0000 7fc5b21fc700 1 Processor -- start 2023-10-07T07:57:54.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.652+0000 7fc5b21fc700 1 -- start start 2023-10-07T07:57:54.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.653+0000 7fc5b21fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5ac063230 0x7fc5ac06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.653+0000 7fc5b21fc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc5ac0c9ec0 0x7fc5ac06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.653+0000 7fc5b21fc700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac06e590 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.653+0000 7fc5b21fc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac0b03a0 con 0x7fc5ac063230 2023-10-07T07:57:54.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.653+0000 7fc5b21fc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac15a330 con 0x7fc5ac0c9ec0 2023-10-07T07:57:54.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.653+0000 7fc5aaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5ac063230 0x7fc5ac06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.653+0000 7fc5aaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5ac063230 0x7fc5ac06db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33514/0 (socket says 172.21.15.7:33514) 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.653+0000 7fc5aaffd700 1 -- 172.21.15.7:0/1440063711 learned_addr learned my addr 172.21.15.7:0/1440063711 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.653+0000 7fc5abfff700 1 --2- 172.21.15.7:0/1440063711 >> v2:172.21.15.172:3300/0 conn(0x7fc5ac0c9ec0 0x7fc5ac06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5aaffd700 1 -- 172.21.15.7:0/1440063711 >> v2:172.21.15.172:3300/0 conn(0x7fc5ac0c9ec0 msgr2=0x7fc5ac06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5aaffd700 1 --2- 172.21.15.7:0/1440063711 >> v2:172.21.15.172:3300/0 conn(0x7fc5ac0c9ec0 0x7fc5ac06e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5aaffd700 1 -- 172.21.15.7:0/1440063711 >> v1:172.21.15.172:6789/0 conn(0x7fc5ac0c9af0 legacy=0x7fc5ac15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5aaffd700 1 -- 172.21.15.7:0/1440063711 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac15a610 con 0x7fc5ac063230 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5aaffd700 1 --2- 172.21.15.7:0/1440063711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac063230 0x7fc5ac06db10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0ae6e0 tx=0x7fc5a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3728b1744492a50e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/1440063711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0009e70 con 0x7fc5ac063230 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/1440063711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a000b860 con 0x7fc5ac063230 2023-10-07T07:57:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5aa7fc700 1 -- 172.21.15.7:0/1440063711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000bc00 con 0x7fc5ac063230 2023-10-07T07:57:54.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1440063711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac063230 msgr2=0x7fc5ac06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.654+0000 7fc5b21fc700 1 --2- 172.21.15.7:0/1440063711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac063230 0x7fc5ac06db10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0ae6e0 tx=0x7fc5a0009250 comp rx=0 tx=0).stop 2023-10-07T07:57:54.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.655+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1440063711 shutdown_connections 2023-10-07T07:57:54.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.655+0000 7fc5b21fc700 1 --2- 172.21.15.7:0/1440063711 >> v2:172.21.15.172:3300/0 conn(0x7fc5ac0c9ec0 0x7fc5ac06e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.655+0000 7fc5b21fc700 1 --2- 172.21.15.7:0/1440063711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac063230 0x7fc5ac06db10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.655+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1440063711 >> 172.21.15.7:0/1440063711 conn(0x7fc5ac0cd1a0 msgr2=0x7fc5ac0c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:54.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.655+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1440063711 shutdown_connections 2023-10-07T07:57:54.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.655+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1440063711 wait complete. 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.656+0000 7fc5b21fc700 1 Processor -- start 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.656+0000 7fc5b21fc700 1 -- start start 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5b21fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5ac063230 0x7fc5ac0c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5b21fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac0c9af0 0x7fc5ac0a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5b21fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5ac0c9ec0 0x7fc5ac14af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5b21fc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0c6270 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5b21fc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0c6550 con 0x7fc5ac063230 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5b21fc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0c6830 con 0x7fc5ac0c9ec0 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac0c9af0 0x7fc5ac0a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac0c9af0 0x7fc5ac0a7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33522/0 (socket says 172.21.15.7:33522) 2023-10-07T07:57:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5aaffd700 1 -- 172.21.15.7:0/1740284285 learned_addr learned my addr 172.21.15.7:0/1740284285 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5abfff700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5ac0c9ec0 0x7fc5ac14af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5ab7fe700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5ac063230 0x7fc5ac0c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5aaffd700 1 -- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5ac0c9ec0 msgr2=0x7fc5ac14af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5aaffd700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5ac0c9ec0 0x7fc5ac14af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5aaffd700 1 -- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5ac063230 msgr2=0x7fc5ac0c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5aaffd700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5ac063230 0x7fc5ac0c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5aaffd700 1 -- 172.21.15.7:0/1740284285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a00094f0 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.657+0000 7fc5aaffd700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac0c9af0 0x7fc5ac0a7040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac06d7c0 tx=0x7fc5a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.658+0000 7fc5937fe700 1 -- 172.21.15.7:0/1740284285 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000bbe0 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.658+0000 7fc5937fe700 1 -- 172.21.15.7:0/1740284285 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0011420 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.658+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac0c6b10 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.658+0000 7fc5937fe700 1 -- 172.21.15.7:0/1740284285 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00117c0 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.658+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac0c5e60 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.659+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac0716c0 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.659+0000 7fc5937fe700 1 -- 172.21.15.7:0/1740284285 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc5a0011bf0 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.660+0000 7fc5937fe700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc598066f30 0x7fc5980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.660+0000 7fc5937fe700 1 -- 172.21.15.7:0/1740284285 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc5a0085970 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.660+0000 7fc5ab7fe700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc598066f30 0x7fc5980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.660+0000 7fc5ab7fe700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc598066f30 0x7fc5980693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc59c003a80 tx=0x7fc59c007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:54.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.664+0000 7fc5937fe700 1 -- 172.21.15.7:0/1740284285 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5a0052be0 con 0x7fc5ac0c9af0 2023-10-07T07:57:54.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.862+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5ac0b0780 con 0x7fc598066f30 2023-10-07T07:57:54.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.871+0000 7fc5937fe700 1 -- 172.21.15.7:0/1740284285 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264565 (secure 0 0 0) 0x7fc5ac0b0780 con 0x7fc598066f30 2023-10-07T07:57:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc598066f30 msgr2=0x7fc5980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc598066f30 0x7fc5980693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc59c003a80 tx=0x7fc59c007ec0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac0c9af0 msgr2=0x7fc5ac0a7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac0c9af0 0x7fc5ac0a7040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac06d7c0 tx=0x7fc5a0017000 comp rx=0 tx=0).stop 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 shutdown_connections 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc598066f30 0x7fc5980693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5ac0c9ec0 0x7fc5ac14af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5ac063230 0x7fc5ac0c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 --2- 172.21.15.7:0/1740284285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ac0c9af0 0x7fc5ac0a7040 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:54.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 >> 172.21.15.7:0/1740284285 conn(0x7fc5ac0cd1a0 msgr2=0x7fc5ac1574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:54.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 shutdown_connections 2023-10-07T07:57:54.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:54.874+0000 7fc5b21fc700 1 -- 172.21.15.7:0/1740284285 wait complete. 2023-10-07T07:57:54.878 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T07:57:54.888 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-07T07:57:54.025817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381164,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593556,"statfs":{"total":773094113280,"available":770655801344,"internally_reserved":0,"allocated":2928640,"data_stored":1756412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.331206"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733747+0000","last_change":"2023-10-07T07:57:50.730298+0000","last_active":"2023-10-07T07:57:50.733747+0000","last_peered":"2023-10-07T07:57:50.733747+0000","last_clean":"2023-10-07T07:57:50.733747+0000","last_became_active":"2023-10-07T07:57:50.729981+0000","last_became_peered":"2023-10-07T07:57:50.729981+0000","last_unstale":"2023-10-07T07:57:50.733747+0000","last_undegraded":"2023-10-07T07:57:50.733747+0000","last_fullsized":"2023-10-07T07:57:50.733747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:13:37.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734352+0000","last_change":"2023-10-07T07:57:50.726555+0000","last_active":"2023-10-07T07:57:50.734352+0000","last_peered":"2023-10-07T07:57:50.734352+0000","last_clean":"2023-10-07T07:57:50.734352+0000","last_became_active":"2023-10-07T07:57:50.726036+0000","last_became_peered":"2023-10-07T07:57:50.726036+0000","last_unstale":"2023-10-07T07:57:50.734352+0000","last_undegraded":"2023-10-07T07:57:50.734352+0000","last_fullsized":"2023-10-07T07:57:50.734352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:03:06.838727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726644+0000","last_change":"2023-10-07T07:57:50.722937+0000","last_active":"2023-10-07T07:57:50.726644+0000","last_peered":"2023-10-07T07:57:50.726644+0000","last_clean":"2023-10-07T07:57:50.726644+0000","last_became_active":"2023-10-07T07:57:50.722662+0000","last_became_peered":"2023-10-07T07:57:50.722662+0000","last_unstale":"2023-10-07T07:57:50.726644+0000","last_undegraded":"2023-10-07T07:57:50.726644+0000","last_fullsized":"2023-10-07T07:57:50.726644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:15:16.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725792+0000","last_change":"2023-10-07T07:57:50.725792+0000","last_active":"2023-10-07T07:57:50.725792+0000","last_peered":"2023-10-07T07:57:50.725792+0000","last_clean":"2023-10-07T07:57:50.725792+0000","last_became_active":"2023-10-07T07:57:50.725545+0000","last_became_peered":"2023-10-07T07:57:50.725545+0000","last_unstale":"2023-10-07T07:57:50.725792+0000","last_undegraded":"2023-10-07T07:57:50.725792+0000","last_fullsized":"2023-10-07T07:57:50.725792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739501+0000","last_change":"2023-10-07T07:57:50.737070+0000","last_active":"2023-10-07T07:57:50.739501+0000","last_peered":"2023-10-07T07:57:50.739501+0000","last_clean":"2023-10-07T07:57:50.739501+0000","last_became_active":"2023-10-07T07:57:50.734411+0000","last_became_peered":"2023-10-07T07:57:50.734411+0000","last_unstale":"2023-10-07T07:57:50.739501+0000","last_undegraded":"2023-10-07T07:57:50.739501+0000","last_fullsized":"2023-10-07T07:57:50.739501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:03:23.294266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.719875+0000","last_change":"2023-10-07T07:57:50.719875+0000","last_active":"2023-10-07T07:57:50.719875+0000","last_peered":"2023-10-07T07:57:50.719875+0000","last_clean":"2023-10-07T07:57:50.719875+0000","last_became_active":"2023-10-07T07:57:50.719322+0000","last_became_peered":"2023-10-07T07:57:50.719322+0000","last_unstale":"2023-10-07T07:57:50.719875+0000","last_undegraded":"2023-10-07T07:57:50.719875+0000","last_fullsized":"2023-10-07T07:57:50.719875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732962+0000","last_change":"2023-10-07T07:57:50.728894+0000","last_active":"2023-10-07T07:57:50.732962+0000","last_peered":"2023-10-07T07:57:50.732962+0000","last_clean":"2023-10-07T07:57:50.732962+0000","last_became_active":"2023-10-07T07:57:50.726313+0000","last_became_peered":"2023-10-07T07:57:50.726313+0000","last_unstale":"2023-10-07T07:57:50.732962+0000","last_undegraded":"2023-10-07T07:57:50.732962+0000","last_fullsized":"2023-10-07T07:57:50.732962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.736796+0000","last_change":"2023-10-07T07:57:50.730368+0000","last_active":"2023-10-07T07:57:50.736796+0000","last_peered":"2023-10-07T07:57:50.736796+0000","last_clean":"2023-10-07T07:57:50.736796+0000","last_became_active":"2023-10-07T07:57:50.730094+0000","last_became_peered":"2023-10-07T07:57:50.730094+0000","last_unstale":"2023-10-07T07:57:50.736796+0000","last_undegraded":"2023-10-07T07:57:50.736796+0000","last_fullsized":"2023-10-07T07:57:50.736796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:53:49.221151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734017+0000","last_change":"2023-10-07T07:57:50.726948+0000","last_active":"2023-10-07T07:57:50.734017+0000","last_peered":"2023-10-07T07:57:50.734017+0000","last_clean":"2023-10-07T07:57:50.734017+0000","last_became_active":"2023-10-07T07:57:50.726622+0000","last_became_peered":"2023-10-07T07:57:50.726622+0000","last_unstale":"2023-10-07T07:57:50.734017+0000","last_undegraded":"2023-10-07T07:57:50.734017+0000","last_fullsized":"2023-10-07T07:57:50.734017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:38:53.980034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.745995+0000","last_change":"2023-10-07T07:57:50.745365+0000","last_active":"2023-10-07T07:57:50.745995+0000","last_peered":"2023-10-07T07:57:50.745995+0000","last_clean":"2023-10-07T07:57:50.745995+0000","last_became_active":"2023-10-07T07:57:50.745061+0000","last_became_peered":"2023-10-07T07:57:50.745061+0000","last_unstale":"2023-10-07T07:57:50.745995+0000","last_undegraded":"2023-10-07T07:57:50.745995+0000","last_fullsized":"2023-10-07T07:57:50.745995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:09:38.557090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733023+0000","last_change":"2023-10-07T07:57:50.729317+0000","last_active":"2023-10-07T07:57:50.733023+0000","last_peered":"2023-10-07T07:57:50.733023+0000","last_clean":"2023-10-07T07:57:50.733023+0000","last_became_active":"2023-10-07T07:57:50.726065+0000","last_became_peered":"2023-10-07T07:57:50.726065+0000","last_unstale":"2023-10-07T07:57:50.733023+0000","last_undegraded":"2023-10-07T07:57:50.733023+0000","last_fullsized":"2023-10-07T07:57:50.733023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:59:47.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735217+0000","last_change":"2023-10-07T07:57:50.734224+0000","last_active":"2023-10-07T07:57:50.735217+0000","last_peered":"2023-10-07T07:57:50.735217+0000","last_clean":"2023-10-07T07:57:50.735217+0000","last_became_active":"2023-10-07T07:57:50.729939+0000","last_became_peered":"2023-10-07T07:57:50.729939+0000","last_unstale":"2023-10-07T07:57:50.735217+0000","last_undegraded":"2023-10-07T07:57:50.735217+0000","last_fullsized":"2023-10-07T07:57:50.735217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:53:49.221151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735896+0000","last_change":"2023-10-07T07:57:50.730055+0000","last_active":"2023-10-07T07:57:50.735896+0000","last_peered":"2023-10-07T07:57:50.735896+0000","last_clean":"2023-10-07T07:57:50.735896+0000","last_became_active":"2023-10-07T07:57:50.729800+0000","last_became_peered":"2023-10-07T07:57:50.729800+0000","last_unstale":"2023-10-07T07:57:50.735896+0000","last_undegraded":"2023-10-07T07:57:50.735896+0000","last_fullsized":"2023-10-07T07:57:50.735896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:37:11.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725487+0000","last_change":"2023-10-07T07:57:50.718932+0000","last_active":"2023-10-07T07:57:50.725487+0000","last_peered":"2023-10-07T07:57:50.725487+0000","last_clean":"2023-10-07T07:57:50.725487+0000","last_became_active":"2023-10-07T07:57:50.718650+0000","last_became_peered":"2023-10-07T07:57:50.718650+0000","last_unstale":"2023-10-07T07:57:50.725487+0000","last_undegraded":"2023-10-07T07:57:50.725487+0000","last_fullsized":"2023-10-07T07:57:50.725487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:40.473036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.737108+0000","last_change":"2023-10-07T07:57:50.730279+0000","last_active":"2023-10-07T07:57:50.737108+0000","last_peered":"2023-10-07T07:57:50.737108+0000","last_clean":"2023-10-07T07:57:50.737108+0000","last_became_active":"2023-10-07T07:57:50.730004+0000","last_became_peered":"2023-10-07T07:57:50.730004+0000","last_unstale":"2023-10-07T07:57:50.737108+0000","last_undegraded":"2023-10-07T07:57:50.737108+0000","last_fullsized":"2023-10-07T07:57:50.737108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.720796+0000","last_change":"2023-10-07T07:57:50.720796+0000","last_active":"2023-10-07T07:57:50.720796+0000","last_peered":"2023-10-07T07:57:50.720796+0000","last_clean":"2023-10-07T07:57:50.720796+0000","last_became_active":"2023-10-07T07:57:50.720416+0000","last_became_peered":"2023-10-07T07:57:50.720416+0000","last_unstale":"2023-10-07T07:57:50.720796+0000","last_undegraded":"2023-10-07T07:57:50.720796+0000","last_fullsized":"2023-10-07T07:57:50.720796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:03:44.885267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741002+0000","last_change":"2023-10-07T07:57:50.741002+0000","last_active":"2023-10-07T07:57:50.741002+0000","last_peered":"2023-10-07T07:57:50.741002+0000","last_clean":"2023-10-07T07:57:50.741002+0000","last_became_active":"2023-10-07T07:57:50.740515+0000","last_became_peered":"2023-10-07T07:57:50.740515+0000","last_unstale":"2023-10-07T07:57:50.741002+0000","last_undegraded":"2023-10-07T07:57:50.741002+0000","last_fullsized":"2023-10-07T07:57:50.741002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:35:45.958299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.731000+0000","last_change":"2023-10-07T07:57:50.725365+0000","last_active":"2023-10-07T07:57:50.731000+0000","last_peered":"2023-10-07T07:57:50.731000+0000","last_clean":"2023-10-07T07:57:50.731000+0000","last_became_active":"2023-10-07T07:57:50.725076+0000","last_became_peered":"2023-10-07T07:57:50.725076+0000","last_unstale":"2023-10-07T07:57:50.731000+0000","last_undegraded":"2023-10-07T07:57:50.731000+0000","last_fullsized":"2023-10-07T07:57:50.731000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722362+0000","last_change":"2023-10-07T07:57:50.722362+0000","last_active":"2023-10-07T07:57:50.722362+0000","last_peered":"2023-10-07T07:57:50.722362+0000","last_clean":"2023-10-07T07:57:50.722362+0000","last_became_active":"2023-10-07T07:57:50.722022+0000","last_became_peered":"2023-10-07T07:57:50.722022+0000","last_unstale":"2023-10-07T07:57:50.722362+0000","last_undegraded":"2023-10-07T07:57:50.722362+0000","last_fullsized":"2023-10-07T07:57:50.722362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:25:31.456924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742832+0000","last_change":"2023-10-07T07:57:49.726945+0000","last_active":"2023-10-07T07:57:50.742832+0000","last_peered":"2023-10-07T07:57:50.742832+0000","last_clean":"2023-10-07T07:57:50.742832+0000","last_became_active":"2023-10-07T07:57:49.726621+0000","last_became_peered":"2023-10-07T07:57:49.726621+0000","last_unstale":"2023-10-07T07:57:50.742832+0000","last_undegraded":"2023-10-07T07:57:50.742832+0000","last_fullsized":"2023-10-07T07:57:50.742832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741522+0000","last_change":"2023-10-07T07:57:50.740835+0000","last_active":"2023-10-07T07:57:50.741522+0000","last_peered":"2023-10-07T07:57:50.741522+0000","last_clean":"2023-10-07T07:57:50.741522+0000","last_became_active":"2023-10-07T07:57:50.740562+0000","last_became_peered":"2023-10-07T07:57:50.740562+0000","last_unstale":"2023-10-07T07:57:50.741522+0000","last_undegraded":"2023-10-07T07:57:50.741522+0000","last_fullsized":"2023-10-07T07:57:50.741522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:09:38.557090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738888+0000","last_change":"2023-10-07T07:57:49.725329+0000","last_active":"2023-10-07T07:57:50.738888+0000","last_peered":"2023-10-07T07:57:50.738888+0000","last_clean":"2023-10-07T07:57:50.738888+0000","last_became_active":"2023-10-07T07:57:49.724930+0000","last_became_peered":"2023-10-07T07:57:49.724930+0000","last_unstale":"2023-10-07T07:57:50.738888+0000","last_undegraded":"2023-10-07T07:57:50.738888+0000","last_fullsized":"2023-10-07T07:57:50.738888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725494+0000","last_change":"2023-10-07T07:57:50.724035+0000","last_active":"2023-10-07T07:57:50.725494+0000","last_peered":"2023-10-07T07:57:50.725494+0000","last_clean":"2023-10-07T07:57:50.725494+0000","last_became_active":"2023-10-07T07:57:50.723418+0000","last_became_peered":"2023-10-07T07:57:50.723418+0000","last_unstale":"2023-10-07T07:57:50.725494+0000","last_undegraded":"2023-10-07T07:57:50.725494+0000","last_fullsized":"2023-10-07T07:57:50.725494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741778+0000","last_change":"2023-10-07T07:57:49.727656+0000","last_active":"2023-10-07T07:57:50.741778+0000","last_peered":"2023-10-07T07:57:50.741778+0000","last_clean":"2023-10-07T07:57:50.741778+0000","last_became_active":"2023-10-07T07:57:49.727380+0000","last_became_peered":"2023-10-07T07:57:49.727380+0000","last_unstale":"2023-10-07T07:57:50.741778+0000","last_undegraded":"2023-10-07T07:57:50.741778+0000","last_fullsized":"2023-10-07T07:57:50.741778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:31.962724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732428+0000","last_change":"2023-10-07T07:57:50.727657+0000","last_active":"2023-10-07T07:57:50.732428+0000","last_peered":"2023-10-07T07:57:50.732428+0000","last_clean":"2023-10-07T07:57:50.732428+0000","last_became_active":"2023-10-07T07:57:50.727309+0000","last_became_peered":"2023-10-07T07:57:50.727309+0000","last_unstale":"2023-10-07T07:57:50.732428+0000","last_undegraded":"2023-10-07T07:57:50.732428+0000","last_fullsized":"2023-10-07T07:57:50.732428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743139+0000","last_change":"2023-10-07T07:57:49.729636+0000","last_active":"2023-10-07T07:57:50.743139+0000","last_peered":"2023-10-07T07:57:50.743139+0000","last_clean":"2023-10-07T07:57:50.743139+0000","last_became_active":"2023-10-07T07:57:49.725076+0000","last_became_peered":"2023-10-07T07:57:49.725076+0000","last_unstale":"2023-10-07T07:57:50.743139+0000","last_undegraded":"2023-10-07T07:57:50.743139+0000","last_fullsized":"2023-10-07T07:57:50.743139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:01.309236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742189+0000","last_change":"2023-10-07T07:57:50.742189+0000","last_active":"2023-10-07T07:57:50.742189+0000","last_peered":"2023-10-07T07:57:50.742189+0000","last_clean":"2023-10-07T07:57:50.742189+0000","last_became_active":"2023-10-07T07:57:50.741823+0000","last_became_peered":"2023-10-07T07:57:50.741823+0000","last_unstale":"2023-10-07T07:57:50.742189+0000","last_undegraded":"2023-10-07T07:57:50.742189+0000","last_fullsized":"2023-10-07T07:57:50.742189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:18:23.885325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738865+0000","last_change":"2023-10-07T07:57:49.724287+0000","last_active":"2023-10-07T07:57:50.738865+0000","last_peered":"2023-10-07T07:57:50.738865+0000","last_clean":"2023-10-07T07:57:50.738865+0000","last_became_active":"2023-10-07T07:57:49.723711+0000","last_became_peered":"2023-10-07T07:57:49.723711+0000","last_unstale":"2023-10-07T07:57:50.738865+0000","last_undegraded":"2023-10-07T07:57:50.738865+0000","last_fullsized":"2023-10-07T07:57:50.738865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726639+0000","last_change":"2023-10-07T07:57:50.723737+0000","last_active":"2023-10-07T07:57:50.726639+0000","last_peered":"2023-10-07T07:57:50.726639+0000","last_clean":"2023-10-07T07:57:50.726639+0000","last_became_active":"2023-10-07T07:57:50.723444+0000","last_became_peered":"2023-10-07T07:57:50.723444+0000","last_unstale":"2023-10-07T07:57:50.726639+0000","last_undegraded":"2023-10-07T07:57:50.726639+0000","last_fullsized":"2023-10-07T07:57:50.726639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712156+0000","last_change":"2023-10-07T07:57:49.737749+0000","last_active":"2023-10-07T07:57:50.712156+0000","last_peered":"2023-10-07T07:57:50.712156+0000","last_clean":"2023-10-07T07:57:50.712156+0000","last_became_active":"2023-10-07T07:57:49.737477+0000","last_became_peered":"2023-10-07T07:57:49.737477+0000","last_unstale":"2023-10-07T07:57:50.712156+0000","last_undegraded":"2023-10-07T07:57:50.712156+0000","last_fullsized":"2023-10-07T07:57:50.712156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:10.280748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.730239+0000","last_change":"2023-10-07T07:57:50.730239+0000","last_active":"2023-10-07T07:57:50.730239+0000","last_peered":"2023-10-07T07:57:50.730239+0000","last_clean":"2023-10-07T07:57:50.730239+0000","last_became_active":"2023-10-07T07:57:50.729972+0000","last_became_peered":"2023-10-07T07:57:50.729972+0000","last_unstale":"2023-10-07T07:57:50.730239+0000","last_undegraded":"2023-10-07T07:57:50.730239+0000","last_fullsized":"2023-10-07T07:57:50.730239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:03:06.838727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711739+0000","last_change":"2023-10-07T07:57:49.728260+0000","last_active":"2023-10-07T07:57:50.711739+0000","last_peered":"2023-10-07T07:57:50.711739+0000","last_clean":"2023-10-07T07:57:50.711739+0000","last_became_active":"2023-10-07T07:57:49.728019+0000","last_became_peered":"2023-10-07T07:57:49.728019+0000","last_unstale":"2023-10-07T07:57:50.711739+0000","last_undegraded":"2023-10-07T07:57:50.711739+0000","last_fullsized":"2023-10-07T07:57:50.711739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726987+0000","last_change":"2023-10-07T07:57:50.723700+0000","last_active":"2023-10-07T07:57:50.726987+0000","last_peered":"2023-10-07T07:57:50.726987+0000","last_clean":"2023-10-07T07:57:50.726987+0000","last_became_active":"2023-10-07T07:57:50.723420+0000","last_became_peered":"2023-10-07T07:57:50.723420+0000","last_unstale":"2023-10-07T07:57:50.726987+0000","last_undegraded":"2023-10-07T07:57:50.726987+0000","last_fullsized":"2023-10-07T07:57:50.726987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:26:00.336772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.736925+0000","last_change":"2023-10-07T07:57:49.732565+0000","last_active":"2023-10-07T07:57:49.736925+0000","last_peered":"2023-10-07T07:57:49.736925+0000","last_clean":"2023-10-07T07:57:49.736925+0000","last_became_active":"2023-10-07T07:57:49.732250+0000","last_became_peered":"2023-10-07T07:57:49.732250+0000","last_unstale":"2023-10-07T07:57:49.736925+0000","last_undegraded":"2023-10-07T07:57:49.736925+0000","last_fullsized":"2023-10-07T07:57:49.736925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722227+0000","last_change":"2023-10-07T07:57:50.722227+0000","last_active":"2023-10-07T07:57:50.722227+0000","last_peered":"2023-10-07T07:57:50.722227+0000","last_clean":"2023-10-07T07:57:50.722227+0000","last_became_active":"2023-10-07T07:57:50.721953+0000","last_became_peered":"2023-10-07T07:57:50.721953+0000","last_unstale":"2023-10-07T07:57:50.722227+0000","last_undegraded":"2023-10-07T07:57:50.722227+0000","last_fullsized":"2023-10-07T07:57:50.722227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:50:56.892918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.709932+0000","last_change":"2023-10-07T07:57:49.720571+0000","last_active":"2023-10-07T07:57:50.709932+0000","last_peered":"2023-10-07T07:57:50.709932+0000","last_clean":"2023-10-07T07:57:50.709932+0000","last_became_active":"2023-10-07T07:57:49.720226+0000","last_became_peered":"2023-10-07T07:57:49.720226+0000","last_unstale":"2023-10-07T07:57:50.709932+0000","last_undegraded":"2023-10-07T07:57:50.709932+0000","last_fullsized":"2023-10-07T07:57:50.709932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724121+0000","last_change":"2023-10-07T07:57:50.724121+0000","last_active":"2023-10-07T07:57:50.724121+0000","last_peered":"2023-10-07T07:57:50.724121+0000","last_clean":"2023-10-07T07:57:50.724121+0000","last_became_active":"2023-10-07T07:57:50.723789+0000","last_became_peered":"2023-10-07T07:57:50.723789+0000","last_unstale":"2023-10-07T07:57:50.724121+0000","last_undegraded":"2023-10-07T07:57:50.724121+0000","last_fullsized":"2023-10-07T07:57:50.724121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.720435+0000","last_change":"2023-10-07T07:57:49.733414+0000","last_active":"2023-10-07T07:57:50.720435+0000","last_peered":"2023-10-07T07:57:50.720435+0000","last_clean":"2023-10-07T07:57:50.720435+0000","last_became_active":"2023-10-07T07:57:49.732736+0000","last_became_peered":"2023-10-07T07:57:49.732736+0000","last_unstale":"2023-10-07T07:57:50.720435+0000","last_undegraded":"2023-10-07T07:57:50.720435+0000","last_fullsized":"2023-10-07T07:57:50.720435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:49.692513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727348+0000","last_change":"2023-10-07T07:57:50.724743+0000","last_active":"2023-10-07T07:57:50.727348+0000","last_peered":"2023-10-07T07:57:50.727348+0000","last_clean":"2023-10-07T07:57:50.727348+0000","last_became_active":"2023-10-07T07:57:50.724501+0000","last_became_peered":"2023-10-07T07:57:50.724501+0000","last_unstale":"2023-10-07T07:57:50.727348+0000","last_undegraded":"2023-10-07T07:57:50.727348+0000","last_fullsized":"2023-10-07T07:57:50.727348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:03:23.294266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738823+0000","last_change":"2023-10-07T07:57:49.721843+0000","last_active":"2023-10-07T07:57:50.738823+0000","last_peered":"2023-10-07T07:57:50.738823+0000","last_clean":"2023-10-07T07:57:50.738823+0000","last_became_active":"2023-10-07T07:57:49.721533+0000","last_became_peered":"2023-10-07T07:57:49.721533+0000","last_unstale":"2023-10-07T07:57:50.738823+0000","last_undegraded":"2023-10-07T07:57:50.738823+0000","last_fullsized":"2023-10-07T07:57:50.738823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724096+0000","last_change":"2023-10-07T07:57:50.724096+0000","last_active":"2023-10-07T07:57:50.724096+0000","last_peered":"2023-10-07T07:57:50.724096+0000","last_clean":"2023-10-07T07:57:50.724096+0000","last_became_active":"2023-10-07T07:57:50.723782+0000","last_became_peered":"2023-10-07T07:57:50.723782+0000","last_unstale":"2023-10-07T07:57:50.724096+0000","last_undegraded":"2023-10-07T07:57:50.724096+0000","last_fullsized":"2023-10-07T07:57:50.724096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:59:47.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742091+0000","last_change":"2023-10-07T07:57:49.720185+0000","last_active":"2023-10-07T07:57:50.742091+0000","last_peered":"2023-10-07T07:57:50.742091+0000","last_clean":"2023-10-07T07:57:50.742091+0000","last_became_active":"2023-10-07T07:57:49.719880+0000","last_became_peered":"2023-10-07T07:57:49.719880+0000","last_unstale":"2023-10-07T07:57:50.742091+0000","last_undegraded":"2023-10-07T07:57:50.742091+0000","last_fullsized":"2023-10-07T07:57:50.742091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:53:25.421958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732557+0000","last_change":"2023-10-07T07:57:50.726280+0000","last_active":"2023-10-07T07:57:50.732557+0000","last_peered":"2023-10-07T07:57:50.732557+0000","last_clean":"2023-10-07T07:57:50.732557+0000","last_became_active":"2023-10-07T07:57:50.725649+0000","last_became_peered":"2023-10-07T07:57:50.725649+0000","last_unstale":"2023-10-07T07:57:50.732557+0000","last_undegraded":"2023-10-07T07:57:50.732557+0000","last_fullsized":"2023-10-07T07:57:50.732557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:22:50.873696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739735+0000","last_change":"2023-10-07T07:57:49.732331+0000","last_active":"2023-10-07T07:57:50.739735+0000","last_peered":"2023-10-07T07:57:50.739735+0000","last_clean":"2023-10-07T07:57:50.739735+0000","last_became_active":"2023-10-07T07:57:49.732085+0000","last_became_peered":"2023-10-07T07:57:49.732085+0000","last_unstale":"2023-10-07T07:57:50.739735+0000","last_undegraded":"2023-10-07T07:57:50.739735+0000","last_fullsized":"2023-10-07T07:57:50.739735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727938+0000","last_change":"2023-10-07T07:57:50.724196+0000","last_active":"2023-10-07T07:57:50.727938+0000","last_peered":"2023-10-07T07:57:50.727938+0000","last_clean":"2023-10-07T07:57:50.727938+0000","last_became_active":"2023-10-07T07:57:50.723911+0000","last_became_peered":"2023-10-07T07:57:50.723911+0000","last_unstale":"2023-10-07T07:57:50.727938+0000","last_undegraded":"2023-10-07T07:57:50.727938+0000","last_fullsized":"2023-10-07T07:57:50.727938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:42.757604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737857+0000","last_change":"2023-10-07T07:57:49.734820+0000","last_active":"2023-10-07T07:57:49.737857+0000","last_peered":"2023-10-07T07:57:49.737857+0000","last_clean":"2023-10-07T07:57:49.737857+0000","last_became_active":"2023-10-07T07:57:49.732635+0000","last_became_peered":"2023-10-07T07:57:49.732635+0000","last_unstale":"2023-10-07T07:57:49.737857+0000","last_undegraded":"2023-10-07T07:57:49.737857+0000","last_fullsized":"2023-10-07T07:57:49.737857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:36:22.098994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740602+0000","last_change":"2023-10-07T07:57:50.740602+0000","last_active":"2023-10-07T07:57:50.740602+0000","last_peered":"2023-10-07T07:57:50.740602+0000","last_clean":"2023-10-07T07:57:50.740602+0000","last_became_active":"2023-10-07T07:57:50.740273+0000","last_became_peered":"2023-10-07T07:57:50.740273+0000","last_unstale":"2023-10-07T07:57:50.740602+0000","last_undegraded":"2023-10-07T07:57:50.740602+0000","last_fullsized":"2023-10-07T07:57:50.740602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:33:31.464165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.716494+0000","last_change":"2023-10-07T07:57:49.738688+0000","last_active":"2023-10-07T07:57:50.716494+0000","last_peered":"2023-10-07T07:57:50.716494+0000","last_clean":"2023-10-07T07:57:50.716494+0000","last_became_active":"2023-10-07T07:57:49.738387+0000","last_became_peered":"2023-10-07T07:57:49.738387+0000","last_unstale":"2023-10-07T07:57:50.716494+0000","last_undegraded":"2023-10-07T07:57:50.716494+0000","last_fullsized":"2023-10-07T07:57:50.716494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726565+0000","last_change":"2023-10-07T07:57:50.723301+0000","last_active":"2023-10-07T07:57:50.726565+0000","last_peered":"2023-10-07T07:57:50.726565+0000","last_clean":"2023-10-07T07:57:50.726565+0000","last_became_active":"2023-10-07T07:57:50.722920+0000","last_became_peered":"2023-10-07T07:57:50.722920+0000","last_unstale":"2023-10-07T07:57:50.726565+0000","last_undegraded":"2023-10-07T07:57:50.726565+0000","last_fullsized":"2023-10-07T07:57:50.726565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:32.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739521+0000","last_change":"2023-10-07T07:57:49.719344+0000","last_active":"2023-10-07T07:57:50.739521+0000","last_peered":"2023-10-07T07:57:50.739521+0000","last_clean":"2023-10-07T07:57:50.739521+0000","last_became_active":"2023-10-07T07:57:49.719053+0000","last_became_peered":"2023-10-07T07:57:49.719053+0000","last_unstale":"2023-10-07T07:57:50.739521+0000","last_undegraded":"2023-10-07T07:57:50.739521+0000","last_fullsized":"2023-10-07T07:57:50.739521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:32:15.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724500+0000","last_change":"2023-10-07T07:57:50.724500+0000","last_active":"2023-10-07T07:57:50.724500+0000","last_peered":"2023-10-07T07:57:50.724500+0000","last_clean":"2023-10-07T07:57:50.724500+0000","last_became_active":"2023-10-07T07:57:50.724175+0000","last_became_peered":"2023-10-07T07:57:50.724175+0000","last_unstale":"2023-10-07T07:57:50.724500+0000","last_undegraded":"2023-10-07T07:57:50.724500+0000","last_fullsized":"2023-10-07T07:57:50.724500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739659+0000","last_change":"2023-10-07T07:57:49.733130+0000","last_active":"2023-10-07T07:57:50.739659+0000","last_peered":"2023-10-07T07:57:50.739659+0000","last_clean":"2023-10-07T07:57:50.739659+0000","last_became_active":"2023-10-07T07:57:49.732810+0000","last_became_peered":"2023-10-07T07:57:49.732810+0000","last_unstale":"2023-10-07T07:57:50.739659+0000","last_undegraded":"2023-10-07T07:57:50.739659+0000","last_fullsized":"2023-10-07T07:57:50.739659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:32:15.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.731887+0000","last_change":"2023-10-07T07:57:50.731887+0000","last_active":"2023-10-07T07:57:50.731887+0000","last_peered":"2023-10-07T07:57:50.731887+0000","last_clean":"2023-10-07T07:57:50.731887+0000","last_became_active":"2023-10-07T07:57:50.731535+0000","last_became_peered":"2023-10-07T07:57:50.731535+0000","last_unstale":"2023-10-07T07:57:50.731887+0000","last_undegraded":"2023-10-07T07:57:50.731887+0000","last_fullsized":"2023-10-07T07:57:50.731887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:40.473036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711292+0000","last_change":"2023-10-07T07:57:49.722522+0000","last_active":"2023-10-07T07:57:50.711292+0000","last_peered":"2023-10-07T07:57:50.711292+0000","last_clean":"2023-10-07T07:57:50.711292+0000","last_became_active":"2023-10-07T07:57:49.722225+0000","last_became_peered":"2023-10-07T07:57:49.722225+0000","last_unstale":"2023-10-07T07:57:50.711292+0000","last_undegraded":"2023-10-07T07:57:50.711292+0000","last_fullsized":"2023-10-07T07:57:50.711292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.717165+0000","last_change":"2023-10-07T07:57:50.717165+0000","last_active":"2023-10-07T07:57:50.717165+0000","last_peered":"2023-10-07T07:57:50.717165+0000","last_clean":"2023-10-07T07:57:50.717165+0000","last_became_active":"2023-10-07T07:57:50.716840+0000","last_became_peered":"2023-10-07T07:57:50.716840+0000","last_unstale":"2023-10-07T07:57:50.717165+0000","last_undegraded":"2023-10-07T07:57:50.717165+0000","last_fullsized":"2023-10-07T07:57:50.717165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:50.698077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737868+0000","last_change":"2023-10-07T07:57:49.735497+0000","last_active":"2023-10-07T07:57:49.737868+0000","last_peered":"2023-10-07T07:57:49.737868+0000","last_clean":"2023-10-07T07:57:49.737868+0000","last_became_active":"2023-10-07T07:57:49.735195+0000","last_became_peered":"2023-10-07T07:57:49.735195+0000","last_unstale":"2023-10-07T07:57:49.737868+0000","last_undegraded":"2023-10-07T07:57:49.737868+0000","last_fullsized":"2023-10-07T07:57:49.737868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:39.467472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742402+0000","last_change":"2023-10-07T07:57:50.741594+0000","last_active":"2023-10-07T07:57:50.742402+0000","last_peered":"2023-10-07T07:57:50.742402+0000","last_clean":"2023-10-07T07:57:50.742402+0000","last_became_active":"2023-10-07T07:57:50.741331+0000","last_became_peered":"2023-10-07T07:57:50.741331+0000","last_unstale":"2023-10-07T07:57:50.742402+0000","last_undegraded":"2023-10-07T07:57:50.742402+0000","last_fullsized":"2023-10-07T07:57:50.742402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:26:00.336772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733078+0000","last_change":"2023-10-07T07:57:50.726796+0000","last_active":"2023-10-07T07:57:50.733078+0000","last_peered":"2023-10-07T07:57:50.733078+0000","last_clean":"2023-10-07T07:57:50.733078+0000","last_became_active":"2023-10-07T07:57:50.725206+0000","last_became_peered":"2023-10-07T07:57:50.725206+0000","last_unstale":"2023-10-07T07:57:50.733078+0000","last_undegraded":"2023-10-07T07:57:50.733078+0000","last_fullsized":"2023-10-07T07:57:50.733078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:46:16.424520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.714634+0000","last_change":"2023-10-07T07:57:49.722494+0000","last_active":"2023-10-07T07:57:50.714634+0000","last_peered":"2023-10-07T07:57:50.714634+0000","last_clean":"2023-10-07T07:57:50.714634+0000","last_became_active":"2023-10-07T07:57:49.722237+0000","last_became_peered":"2023-10-07T07:57:49.722237+0000","last_unstale":"2023-10-07T07:57:50.714634+0000","last_undegraded":"2023-10-07T07:57:50.714634+0000","last_fullsized":"2023-10-07T07:57:50.714634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.718217+0000","last_change":"2023-10-07T07:57:50.718217+0000","last_active":"2023-10-07T07:57:50.718217+0000","last_peered":"2023-10-07T07:57:50.718217+0000","last_clean":"2023-10-07T07:57:50.718217+0000","last_became_active":"2023-10-07T07:57:50.717791+0000","last_became_peered":"2023-10-07T07:57:50.717791+0000","last_unstale":"2023-10-07T07:57:50.718217+0000","last_undegraded":"2023-10-07T07:57:50.718217+0000","last_fullsized":"2023-10-07T07:57:50.718217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:42.757604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737185+0000","last_change":"2023-10-07T07:57:49.733338+0000","last_active":"2023-10-07T07:57:49.737185+0000","last_peered":"2023-10-07T07:57:49.737185+0000","last_clean":"2023-10-07T07:57:49.737185+0000","last_became_active":"2023-10-07T07:57:49.732478+0000","last_became_peered":"2023-10-07T07:57:49.732478+0000","last_unstale":"2023-10-07T07:57:49.737185+0000","last_undegraded":"2023-10-07T07:57:49.737185+0000","last_fullsized":"2023-10-07T07:57:49.737185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:01.309236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.709777+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T07:57:50.709777+0000","last_peered":"2023-10-07T07:57:50.709777+0000","last_clean":"2023-10-07T07:57:50.709777+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T07:57:50.709777+0000","last_undegraded":"2023-10-07T07:57:50.709777+0000","last_fullsized":"2023-10-07T07:57:50.709777+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727905+0000","last_change":"2023-10-07T07:57:50.723683+0000","last_active":"2023-10-07T07:57:50.727905+0000","last_peered":"2023-10-07T07:57:50.727905+0000","last_clean":"2023-10-07T07:57:50.727905+0000","last_became_active":"2023-10-07T07:57:50.723408+0000","last_became_peered":"2023-10-07T07:57:50.723408+0000","last_unstale":"2023-10-07T07:57:50.727905+0000","last_undegraded":"2023-10-07T07:57:50.727905+0000","last_fullsized":"2023-10-07T07:57:50.727905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:11.286312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737533+0000","last_change":"2023-10-07T07:57:49.733672+0000","last_active":"2023-10-07T07:57:49.737533+0000","last_peered":"2023-10-07T07:57:49.737533+0000","last_clean":"2023-10-07T07:57:49.737533+0000","last_became_active":"2023-10-07T07:57:49.733375+0000","last_became_peered":"2023-10-07T07:57:49.733375+0000","last_unstale":"2023-10-07T07:57:49.737533+0000","last_undegraded":"2023-10-07T07:57:49.737533+0000","last_fullsized":"2023-10-07T07:57:49.737533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.736882+0000","last_change":"2023-10-07T07:57:49.730927+0000","last_active":"2023-10-07T07:57:49.736882+0000","last_peered":"2023-10-07T07:57:49.736882+0000","last_clean":"2023-10-07T07:57:49.736882+0000","last_became_active":"2023-10-07T07:57:49.727008+0000","last_became_peered":"2023-10-07T07:57:49.727008+0000","last_unstale":"2023-10-07T07:57:49.736882+0000","last_undegraded":"2023-10-07T07:57:49.736882+0000","last_fullsized":"2023-10-07T07:57:49.736882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727827+0000","last_change":"2023-10-07T07:57:50.723292+0000","last_active":"2023-10-07T07:57:50.727827+0000","last_peered":"2023-10-07T07:57:50.727827+0000","last_clean":"2023-10-07T07:57:50.727827+0000","last_became_active":"2023-10-07T07:57:50.722388+0000","last_became_peered":"2023-10-07T07:57:50.722388+0000","last_unstale":"2023-10-07T07:57:50.727827+0000","last_undegraded":"2023-10-07T07:57:50.727827+0000","last_fullsized":"2023-10-07T07:57:50.727827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:53:57.122070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.740394+0000","last_change":"2023-10-07T07:57:49.736910+0000","last_active":"2023-10-07T07:57:49.740394+0000","last_peered":"2023-10-07T07:57:49.740394+0000","last_clean":"2023-10-07T07:57:49.740394+0000","last_became_active":"2023-10-07T07:57:49.736645+0000","last_became_peered":"2023-10-07T07:57:49.736645+0000","last_unstale":"2023-10-07T07:57:49.740394+0000","last_undegraded":"2023-10-07T07:57:49.740394+0000","last_fullsized":"2023-10-07T07:57:49.740394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.721660+0000","last_change":"2023-10-07T07:57:50.721660+0000","last_active":"2023-10-07T07:57:50.721660+0000","last_peered":"2023-10-07T07:57:50.721660+0000","last_clean":"2023-10-07T07:57:50.721660+0000","last_became_active":"2023-10-07T07:57:50.721390+0000","last_became_peered":"2023-10-07T07:57:50.721390+0000","last_unstale":"2023-10-07T07:57:50.721660+0000","last_undegraded":"2023-10-07T07:57:50.721660+0000","last_fullsized":"2023-10-07T07:57:50.721660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:11.286312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.745147+0000","last_change":"2023-10-07T07:57:49.737869+0000","last_active":"2023-10-07T07:57:50.745147+0000","last_peered":"2023-10-07T07:57:50.745147+0000","last_clean":"2023-10-07T07:57:50.745147+0000","last_became_active":"2023-10-07T07:57:49.737571+0000","last_became_peered":"2023-10-07T07:57:49.737571+0000","last_unstale":"2023-10-07T07:57:50.745147+0000","last_undegraded":"2023-10-07T07:57:50.745147+0000","last_fullsized":"2023-10-07T07:57:50.745147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:49.692513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.718352+0000","last_change":"2023-10-07T07:57:50.718352+0000","last_active":"2023-10-07T07:57:50.718352+0000","last_peered":"2023-10-07T07:57:50.718352+0000","last_clean":"2023-10-07T07:57:50.718352+0000","last_became_active":"2023-10-07T07:57:50.718091+0000","last_became_peered":"2023-10-07T07:57:50.718091+0000","last_unstale":"2023-10-07T07:57:50.718352+0000","last_undegraded":"2023-10-07T07:57:50.718352+0000","last_fullsized":"2023-10-07T07:57:50.718352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:37:11.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.714964+0000","last_change":"2023-10-07T07:57:49.730601+0000","last_active":"2023-10-07T07:57:50.714964+0000","last_peered":"2023-10-07T07:57:50.714964+0000","last_clean":"2023-10-07T07:57:50.714964+0000","last_became_active":"2023-10-07T07:57:49.730370+0000","last_became_peered":"2023-10-07T07:57:49.730370+0000","last_unstale":"2023-10-07T07:57:50.714964+0000","last_undegraded":"2023-10-07T07:57:50.714964+0000","last_fullsized":"2023-10-07T07:57:50.714964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.723162+0000","last_change":"2023-10-07T07:57:50.723162+0000","last_active":"2023-10-07T07:57:50.723162+0000","last_peered":"2023-10-07T07:57:50.723162+0000","last_clean":"2023-10-07T07:57:50.723162+0000","last_became_active":"2023-10-07T07:57:50.722848+0000","last_became_peered":"2023-10-07T07:57:50.722848+0000","last_unstale":"2023-10-07T07:57:50.723162+0000","last_undegraded":"2023-10-07T07:57:50.723162+0000","last_fullsized":"2023-10-07T07:57:50.723162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737608+0000","last_change":"2023-10-07T07:57:49.731859+0000","last_active":"2023-10-07T07:57:49.737608+0000","last_peered":"2023-10-07T07:57:49.737608+0000","last_clean":"2023-10-07T07:57:49.737608+0000","last_became_active":"2023-10-07T07:57:49.731613+0000","last_became_peered":"2023-10-07T07:57:49.731613+0000","last_unstale":"2023-10-07T07:57:49.737608+0000","last_undegraded":"2023-10-07T07:57:49.737608+0000","last_fullsized":"2023-10-07T07:57:49.737608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.723167+0000","last_change":"2023-10-07T07:57:50.723167+0000","last_active":"2023-10-07T07:57:50.723167+0000","last_peered":"2023-10-07T07:57:50.723167+0000","last_clean":"2023-10-07T07:57:50.723167+0000","last_became_active":"2023-10-07T07:57:50.722863+0000","last_became_peered":"2023-10-07T07:57:50.722863+0000","last_unstale":"2023-10-07T07:57:50.723167+0000","last_undegraded":"2023-10-07T07:57:50.723167+0000","last_fullsized":"2023-10-07T07:57:50.723167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711876+0000","last_change":"2023-10-07T07:57:49.725087+0000","last_active":"2023-10-07T07:57:50.711876+0000","last_peered":"2023-10-07T07:57:50.711876+0000","last_clean":"2023-10-07T07:57:50.711876+0000","last_became_active":"2023-10-07T07:57:49.724027+0000","last_became_peered":"2023-10-07T07:57:49.724027+0000","last_unstale":"2023-10-07T07:57:50.711876+0000","last_undegraded":"2023-10-07T07:57:50.711876+0000","last_fullsized":"2023-10-07T07:57:50.711876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722299+0000","last_change":"2023-10-07T07:57:50.717348+0000","last_active":"2023-10-07T07:57:50.722299+0000","last_peered":"2023-10-07T07:57:50.722299+0000","last_clean":"2023-10-07T07:57:50.722299+0000","last_became_active":"2023-10-07T07:57:50.716959+0000","last_became_peered":"2023-10-07T07:57:50.716959+0000","last_unstale":"2023-10-07T07:57:50.722299+0000","last_undegraded":"2023-10-07T07:57:50.722299+0000","last_fullsized":"2023-10-07T07:57:50.722299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742034+0000","last_change":"2023-10-07T07:57:49.732599+0000","last_active":"2023-10-07T07:57:50.742034+0000","last_peered":"2023-10-07T07:57:50.742034+0000","last_clean":"2023-10-07T07:57:50.742034+0000","last_became_active":"2023-10-07T07:57:49.732088+0000","last_became_peered":"2023-10-07T07:57:49.732088+0000","last_unstale":"2023-10-07T07:57:50.742034+0000","last_undegraded":"2023-10-07T07:57:50.742034+0000","last_fullsized":"2023-10-07T07:57:50.742034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:31.103335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725007+0000","last_change":"2023-10-07T07:57:50.725007+0000","last_active":"2023-10-07T07:57:50.725007+0000","last_peered":"2023-10-07T07:57:50.725007+0000","last_clean":"2023-10-07T07:57:50.725007+0000","last_became_active":"2023-10-07T07:57:50.723895+0000","last_became_peered":"2023-10-07T07:57:50.723895+0000","last_unstale":"2023-10-07T07:57:50.725007+0000","last_undegraded":"2023-10-07T07:57:50.725007+0000","last_fullsized":"2023-10-07T07:57:50.725007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:13:37.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712397+0000","last_change":"2023-10-07T07:57:49.725916+0000","last_active":"2023-10-07T07:57:50.712397+0000","last_peered":"2023-10-07T07:57:50.712397+0000","last_clean":"2023-10-07T07:57:50.712397+0000","last_became_active":"2023-10-07T07:57:49.725648+0000","last_became_peered":"2023-10-07T07:57:49.725648+0000","last_unstale":"2023-10-07T07:57:50.712397+0000","last_undegraded":"2023-10-07T07:57:50.712397+0000","last_fullsized":"2023-10-07T07:57:50.712397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.720453+0000","last_change":"2023-10-07T07:57:50.720453+0000","last_active":"2023-10-07T07:57:50.720453+0000","last_peered":"2023-10-07T07:57:50.720453+0000","last_clean":"2023-10-07T07:57:50.720453+0000","last_became_active":"2023-10-07T07:57:50.719863+0000","last_became_peered":"2023-10-07T07:57:50.719863+0000","last_unstale":"2023-10-07T07:57:50.720453+0000","last_undegraded":"2023-10-07T07:57:50.720453+0000","last_fullsized":"2023-10-07T07:57:50.720453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:21:30.452139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739318+0000","last_change":"2023-10-07T07:57:49.729606+0000","last_active":"2023-10-07T07:57:50.739318+0000","last_peered":"2023-10-07T07:57:50.739318+0000","last_clean":"2023-10-07T07:57:50.739318+0000","last_became_active":"2023-10-07T07:57:49.729372+0000","last_became_peered":"2023-10-07T07:57:49.729372+0000","last_unstale":"2023-10-07T07:57:50.739318+0000","last_undegraded":"2023-10-07T07:57:50.739318+0000","last_fullsized":"2023-10-07T07:57:50.739318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735545+0000","last_change":"2023-10-07T07:57:50.734705+0000","last_active":"2023-10-07T07:57:50.735545+0000","last_peered":"2023-10-07T07:57:50.735545+0000","last_clean":"2023-10-07T07:57:50.735545+0000","last_became_active":"2023-10-07T07:57:50.733706+0000","last_became_peered":"2023-10-07T07:57:50.733706+0000","last_unstale":"2023-10-07T07:57:50.735545+0000","last_undegraded":"2023-10-07T07:57:50.735545+0000","last_fullsized":"2023-10-07T07:57:50.735545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:56:34.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739077+0000","last_change":"2023-10-07T07:57:49.724649+0000","last_active":"2023-10-07T07:57:50.739077+0000","last_peered":"2023-10-07T07:57:50.739077+0000","last_clean":"2023-10-07T07:57:50.739077+0000","last_became_active":"2023-10-07T07:57:49.724368+0000","last_became_peered":"2023-10-07T07:57:49.724368+0000","last_unstale":"2023-10-07T07:57:50.739077+0000","last_undegraded":"2023-10-07T07:57:50.739077+0000","last_fullsized":"2023-10-07T07:57:50.739077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733770+0000","last_change":"2023-10-07T07:57:50.730366+0000","last_active":"2023-10-07T07:57:50.733770+0000","last_peered":"2023-10-07T07:57:50.733770+0000","last_clean":"2023-10-07T07:57:50.733770+0000","last_became_active":"2023-10-07T07:57:50.730012+0000","last_became_peered":"2023-10-07T07:57:50.730012+0000","last_unstale":"2023-10-07T07:57:50.733770+0000","last_undegraded":"2023-10-07T07:57:50.733770+0000","last_fullsized":"2023-10-07T07:57:50.733770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:15:16.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.721978+0000","last_change":"2023-10-07T07:57:50.721978+0000","last_active":"2023-10-07T07:57:50.721978+0000","last_peered":"2023-10-07T07:57:50.721978+0000","last_clean":"2023-10-07T07:57:50.721978+0000","last_became_active":"2023-10-07T07:57:50.719507+0000","last_became_peered":"2023-10-07T07:57:50.719507+0000","last_unstale":"2023-10-07T07:57:50.721978+0000","last_undegraded":"2023-10-07T07:57:50.721978+0000","last_fullsized":"2023-10-07T07:57:50.721978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:32.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.730933+0000","last_change":"2023-10-07T07:57:50.724692+0000","last_active":"2023-10-07T07:57:50.730933+0000","last_peered":"2023-10-07T07:57:50.730933+0000","last_clean":"2023-10-07T07:57:50.730933+0000","last_became_active":"2023-10-07T07:57:50.724411+0000","last_became_peered":"2023-10-07T07:57:50.724411+0000","last_unstale":"2023-10-07T07:57:50.730933+0000","last_undegraded":"2023-10-07T07:57:50.730933+0000","last_fullsized":"2023-10-07T07:57:50.730933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727607+0000","last_change":"2023-10-07T07:57:50.727607+0000","last_active":"2023-10-07T07:57:50.727607+0000","last_peered":"2023-10-07T07:57:50.727607+0000","last_clean":"2023-10-07T07:57:50.727607+0000","last_became_active":"2023-10-07T07:57:50.724048+0000","last_became_peered":"2023-10-07T07:57:50.724048+0000","last_unstale":"2023-10-07T07:57:50.727607+0000","last_undegraded":"2023-10-07T07:57:50.727607+0000","last_fullsized":"2023-10-07T07:57:50.727607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:38:14.739161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734147+0000","last_change":"2023-10-07T07:57:50.726261+0000","last_active":"2023-10-07T07:57:50.734147+0000","last_peered":"2023-10-07T07:57:50.734147+0000","last_clean":"2023-10-07T07:57:50.734147+0000","last_became_active":"2023-10-07T07:57:50.725636+0000","last_became_peered":"2023-10-07T07:57:50.725636+0000","last_unstale":"2023-10-07T07:57:50.734147+0000","last_undegraded":"2023-10-07T07:57:50.734147+0000","last_fullsized":"2023-10-07T07:57:50.734147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:56:34.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738656+0000","last_change":"2023-10-07T07:57:50.735497+0000","last_active":"2023-10-07T07:57:50.738656+0000","last_peered":"2023-10-07T07:57:50.738656+0000","last_clean":"2023-10-07T07:57:50.738656+0000","last_became_active":"2023-10-07T07:57:50.735172+0000","last_became_peered":"2023-10-07T07:57:50.735172+0000","last_unstale":"2023-10-07T07:57:50.738656+0000","last_undegraded":"2023-10-07T07:57:50.738656+0000","last_fullsized":"2023-10-07T07:57:50.738656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725429+0000","last_change":"2023-10-07T07:57:50.718981+0000","last_active":"2023-10-07T07:57:50.725429+0000","last_peered":"2023-10-07T07:57:50.725429+0000","last_clean":"2023-10-07T07:57:50.725429+0000","last_became_active":"2023-10-07T07:57:50.718699+0000","last_became_peered":"2023-10-07T07:57:50.718699+0000","last_unstale":"2023-10-07T07:57:50.725429+0000","last_undegraded":"2023-10-07T07:57:50.725429+0000","last_fullsized":"2023-10-07T07:57:50.725429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:36:23.104558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724708+0000","last_change":"2023-10-07T07:57:50.724708+0000","last_active":"2023-10-07T07:57:50.724708+0000","last_peered":"2023-10-07T07:57:50.724708+0000","last_clean":"2023-10-07T07:57:50.724708+0000","last_became_active":"2023-10-07T07:57:50.724343+0000","last_became_peered":"2023-10-07T07:57:50.724343+0000","last_unstale":"2023-10-07T07:57:50.724708+0000","last_undegraded":"2023-10-07T07:57:50.724708+0000","last_fullsized":"2023-10-07T07:57:50.724708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:38:53.980034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725505+0000","last_change":"2023-10-07T07:57:50.724233+0000","last_active":"2023-10-07T07:57:50.725505+0000","last_peered":"2023-10-07T07:57:50.725505+0000","last_clean":"2023-10-07T07:57:50.725505+0000","last_became_active":"2023-10-07T07:57:50.723770+0000","last_became_peered":"2023-10-07T07:57:50.723770+0000","last_unstale":"2023-10-07T07:57:50.725505+0000","last_undegraded":"2023-10-07T07:57:50.725505+0000","last_fullsized":"2023-10-07T07:57:50.725505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:22:50.873696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724839+0000","last_change":"2023-10-07T07:57:50.724839+0000","last_active":"2023-10-07T07:57:50.724839+0000","last_peered":"2023-10-07T07:57:50.724839+0000","last_clean":"2023-10-07T07:57:50.724839+0000","last_became_active":"2023-10-07T07:57:50.724489+0000","last_became_peered":"2023-10-07T07:57:50.724489+0000","last_unstale":"2023-10-07T07:57:50.724839+0000","last_undegraded":"2023-10-07T07:57:50.724839+0000","last_fullsized":"2023-10-07T07:57:50.724839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:38:11.263837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741272+0000","last_change":"2023-10-07T07:57:50.740752+0000","last_active":"2023-10-07T07:57:50.741272+0000","last_peered":"2023-10-07T07:57:50.741272+0000","last_clean":"2023-10-07T07:57:50.741272+0000","last_became_active":"2023-10-07T07:57:50.740464+0000","last_became_peered":"2023-10-07T07:57:50.740464+0000","last_unstale":"2023-10-07T07:57:50.741272+0000","last_undegraded":"2023-10-07T07:57:50.741272+0000","last_fullsized":"2023-10-07T07:57:50.741272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733824+0000","last_change":"2023-10-07T07:57:50.729375+0000","last_active":"2023-10-07T07:57:50.733824+0000","last_peered":"2023-10-07T07:57:50.733824+0000","last_clean":"2023-10-07T07:57:50.733824+0000","last_became_active":"2023-10-07T07:57:50.729038+0000","last_became_peered":"2023-10-07T07:57:50.729038+0000","last_unstale":"2023-10-07T07:57:50.733824+0000","last_undegraded":"2023-10-07T07:57:50.733824+0000","last_fullsized":"2023-10-07T07:57:50.733824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.718717+0000","last_change":"2023-10-07T07:57:50.718717+0000","last_active":"2023-10-07T07:57:50.718717+0000","last_peered":"2023-10-07T07:57:50.718717+0000","last_clean":"2023-10-07T07:57:50.718717+0000","last_became_active":"2023-10-07T07:57:50.718457+0000","last_became_peered":"2023-10-07T07:57:50.718457+0000","last_unstale":"2023-10-07T07:57:50.718717+0000","last_undegraded":"2023-10-07T07:57:50.718717+0000","last_fullsized":"2023-10-07T07:57:50.718717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:53:57.122070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735843+0000","last_change":"2023-10-07T07:57:50.734975+0000","last_active":"2023-10-07T07:57:50.735843+0000","last_peered":"2023-10-07T07:57:50.735843+0000","last_clean":"2023-10-07T07:57:50.735843+0000","last_became_active":"2023-10-07T07:57:50.734441+0000","last_became_peered":"2023-10-07T07:57:50.734441+0000","last_unstale":"2023-10-07T07:57:50.735843+0000","last_undegraded":"2023-10-07T07:57:50.735843+0000","last_fullsized":"2023-10-07T07:57:50.735843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":659463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":69081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":69579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":71205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298088,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073752,"statfs":{"total":96636764160,"available":96331522048,"internally_reserved":0,"allocated":819200,"data_stored":665773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T07:57:54.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T07:57:55.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.062+0000 7fdeed2e8700 1 Processor -- start 2023-10-07T07:57:55.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.062+0000 7fdeed2e8700 1 -- start start 2023-10-07T07:57:55.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdeed2e8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdee8159380 0x7fdee80b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdeed2e8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee80b4820 con 0x7fdee80b4270 2023-10-07T07:57:55.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdeed2e8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee80b5000 con 0x7fdee80d0250 2023-10-07T07:57:55.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdeed2e8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee80b52e0 con 0x7fdee8159380 2023-10-07T07:57:55.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdee6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdee8159380 0x7fdee80b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdee77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdee80b4270 0x7fdee80ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35934/0 (socket says 172.21.15.7:35934) 2023-10-07T07:57:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdee77fe700 1 -- 172.21.15.7:0/3639403715 learned_addr learned my addr 172.21.15.7:0/3639403715 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdee6ffd700 1 -- 172.21.15.7:0/3639403715 >> v1:172.21.15.172:6789/0 conn(0x7fdee80d0250 legacy=0x7fdee80b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdee6ffd700 1 -- 172.21.15.7:0/3639403715 >> v1:172.21.15.7:6789/0 conn(0x7fdee80b4270 legacy=0x7fdee80ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdee6ffd700 1 -- 172.21.15.7:0/3639403715 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80b55c0 con 0x7fdee8159380 2023-10-07T07:57:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.063+0000 7fdee5ffb700 1 -- 172.21.15.7:0/3639403715 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843104319 0 0) 0x7fdee80b4820 con 0x7fdee80b4270 2023-10-07T07:57:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdee6ffd700 1 --2- 172.21.15.7:0/3639403715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 0x7fdee80b5ae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fded80098e0 tx=0x7fded80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df7b0df691bdf7f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdee5ffb700 1 -- 172.21.15.7:0/3639403715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8009e80 con 0x7fdee8159380 2023-10-07T07:57:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdee5ffb700 1 -- 172.21.15.7:0/3639403715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded800b860 con 0x7fdee8159380 2023-10-07T07:57:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdee5ffb700 1 -- 172.21.15.7:0/3639403715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded800bbe0 con 0x7fdee8159380 2023-10-07T07:57:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdeed2e8700 1 -- 172.21.15.7:0/3639403715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 msgr2=0x7fdee80b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdeed2e8700 1 --2- 172.21.15.7:0/3639403715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 0x7fdee80b5ae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fded80098e0 tx=0x7fded80091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdeed2e8700 1 -- 172.21.15.7:0/3639403715 shutdown_connections 2023-10-07T07:57:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdeed2e8700 1 --2- 172.21.15.7:0/3639403715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 0x7fdee80b5ae0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdeed2e8700 1 -- 172.21.15.7:0/3639403715 >> 172.21.15.7:0/3639403715 conn(0x7fdee8067be0 msgr2=0x7fdee8067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdeed2e8700 1 -- 172.21.15.7:0/3639403715 shutdown_connections 2023-10-07T07:57:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.064+0000 7fdeed2e8700 1 -- 172.21.15.7:0/3639403715 wait complete. 2023-10-07T07:57:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.066+0000 7fdeed2e8700 1 Processor -- start 2023-10-07T07:57:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.066+0000 7fdeed2e8700 1 -- start start 2023-10-07T07:57:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.066+0000 7fdeed2e8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdee80b4270 0x7fdee8151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.066+0000 7fdeed2e8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdee80d0250 0x7fdee8154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdeed2e8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 0x7fdee81546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdeed2e8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8158660 con 0x7fdee8159380 2023-10-07T07:57:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdeed2e8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8151b70 con 0x7fdee80b4270 2023-10-07T07:57:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdeed2e8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8157a50 con 0x7fdee80d0250 2023-10-07T07:57:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 0x7fdee81546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 0x7fdee81546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33544/0 (socket says 172.21.15.7:33544) 2023-10-07T07:57:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee77fe700 1 -- 172.21.15.7:0/1343322555 learned_addr learned my addr 172.21.15.7:0/1343322555 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee6ffd700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdee80b4270 0x7fdee8151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee67fc700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdee80d0250 0x7fdee8154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee77fe700 1 -- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdee80d0250 msgr2=0x7fdee8154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee77fe700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdee80d0250 0x7fdee8154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee77fe700 1 -- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdee80b4270 msgr2=0x7fdee8151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee77fe700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdee80b4270 0x7fdee8151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee77fe700 1 -- 172.21.15.7:0/1343322555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded8009420 con 0x7fdee8159380 2023-10-07T07:57:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdee77fe700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 0x7fdee81546b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdee000cc90 tx=0x7fdee0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdedeffd700 1 -- 172.21.15.7:0/1343322555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee000cea0 con 0x7fdee8159380 2023-10-07T07:57:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdedeffd700 1 -- 172.21.15.7:0/1343322555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee000ab70 con 0x7fdee8159380 2023-10-07T07:57:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdedeffd700 1 -- 172.21.15.7:0/1343322555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee00070f0 con 0x7fdee8159380 2023-10-07T07:57:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee8157d30 con 0x7fdee8159380 2023-10-07T07:57:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.067+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee81582f0 con 0x7fdee8159380 2023-10-07T07:57:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.069+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee80716c0 con 0x7fdee8159380 2023-10-07T07:57:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.069+0000 7fdedeffd700 1 -- 172.21.15.7:0/1343322555 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdee00074a0 con 0x7fdee8159380 2023-10-07T07:57:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.069+0000 7fdedeffd700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdec8066f30 0x7fdec80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.069+0000 7fdedeffd700 1 -- 172.21.15.7:0/1343322555 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdee0084a30 con 0x7fdee8159380 2023-10-07T07:57:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.070+0000 7fdee6ffd700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdec8066f30 0x7fdec80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.070+0000 7fdee6ffd700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdec8066f30 0x7fdec80693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fded80093f0 tx=0x7fded8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.074+0000 7fdedeffd700 1 -- 172.21.15.7:0/1343322555 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdee0051ca0 con 0x7fdee8159380 2023-10-07T07:57:55.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.269+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdee80b0490 con 0x7fdec8066f30 2023-10-07T07:57:55.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.278+0000 7fdedeffd700 1 -- 172.21.15.7:0/1343322555 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264565 (secure 0 0 0) 0x7fdee80b0490 con 0x7fdec8066f30 2023-10-07T07:57:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:55.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdec8066f30 msgr2=0x7fdec80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdec8066f30 0x7fdec80693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fded80093f0 tx=0x7fded8017000 comp rx=0 tx=0).stop 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 msgr2=0x7fdee81546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 0x7fdee81546b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdee000cc90 tx=0x7fdee0001a80 comp rx=0 tx=0).stop 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 shutdown_connections 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdec8066f30 0x7fdec80693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdee8159380 0x7fdee81546b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdee80b4270 0x7fdee8151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 --2- 172.21.15.7:0/1343322555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdee80d0250 0x7fdee8154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.280+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 >> 172.21.15.7:0/1343322555 conn(0x7fdee8067be0 msgr2=0x7fdee8067fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.281+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 shutdown_connections 2023-10-07T07:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.281+0000 7fdeed2e8700 1 -- 172.21.15.7:0/1343322555 wait complete. 2023-10-07T07:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T07:57:55.295 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-07T07:57:54.025817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381164,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593556,"statfs":{"total":773094113280,"available":770655801344,"internally_reserved":0,"allocated":2928640,"data_stored":1756412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.331206"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733747+0000","last_change":"2023-10-07T07:57:50.730298+0000","last_active":"2023-10-07T07:57:50.733747+0000","last_peered":"2023-10-07T07:57:50.733747+0000","last_clean":"2023-10-07T07:57:50.733747+0000","last_became_active":"2023-10-07T07:57:50.729981+0000","last_became_peered":"2023-10-07T07:57:50.729981+0000","last_unstale":"2023-10-07T07:57:50.733747+0000","last_undegraded":"2023-10-07T07:57:50.733747+0000","last_fullsized":"2023-10-07T07:57:50.733747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:13:37.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734352+0000","last_change":"2023-10-07T07:57:50.726555+0000","last_active":"2023-10-07T07:57:50.734352+0000","last_peered":"2023-10-07T07:57:50.734352+0000","last_clean":"2023-10-07T07:57:50.734352+0000","last_became_active":"2023-10-07T07:57:50.726036+0000","last_became_peered":"2023-10-07T07:57:50.726036+0000","last_unstale":"2023-10-07T07:57:50.734352+0000","last_undegraded":"2023-10-07T07:57:50.734352+0000","last_fullsized":"2023-10-07T07:57:50.734352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:03:06.838727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726644+0000","last_change":"2023-10-07T07:57:50.722937+0000","last_active":"2023-10-07T07:57:50.726644+0000","last_peered":"2023-10-07T07:57:50.726644+0000","last_clean":"2023-10-07T07:57:50.726644+0000","last_became_active":"2023-10-07T07:57:50.722662+0000","last_became_peered":"2023-10-07T07:57:50.722662+0000","last_unstale":"2023-10-07T07:57:50.726644+0000","last_undegraded":"2023-10-07T07:57:50.726644+0000","last_fullsized":"2023-10-07T07:57:50.726644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:15:16.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725792+0000","last_change":"2023-10-07T07:57:50.725792+0000","last_active":"2023-10-07T07:57:50.725792+0000","last_peered":"2023-10-07T07:57:50.725792+0000","last_clean":"2023-10-07T07:57:50.725792+0000","last_became_active":"2023-10-07T07:57:50.725545+0000","last_became_peered":"2023-10-07T07:57:50.725545+0000","last_unstale":"2023-10-07T07:57:50.725792+0000","last_undegraded":"2023-10-07T07:57:50.725792+0000","last_fullsized":"2023-10-07T07:57:50.725792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739501+0000","last_change":"2023-10-07T07:57:50.737070+0000","last_active":"2023-10-07T07:57:50.739501+0000","last_peered":"2023-10-07T07:57:50.739501+0000","last_clean":"2023-10-07T07:57:50.739501+0000","last_became_active":"2023-10-07T07:57:50.734411+0000","last_became_peered":"2023-10-07T07:57:50.734411+0000","last_unstale":"2023-10-07T07:57:50.739501+0000","last_undegraded":"2023-10-07T07:57:50.739501+0000","last_fullsized":"2023-10-07T07:57:50.739501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:03:23.294266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.719875+0000","last_change":"2023-10-07T07:57:50.719875+0000","last_active":"2023-10-07T07:57:50.719875+0000","last_peered":"2023-10-07T07:57:50.719875+0000","last_clean":"2023-10-07T07:57:50.719875+0000","last_became_active":"2023-10-07T07:57:50.719322+0000","last_became_peered":"2023-10-07T07:57:50.719322+0000","last_unstale":"2023-10-07T07:57:50.719875+0000","last_undegraded":"2023-10-07T07:57:50.719875+0000","last_fullsized":"2023-10-07T07:57:50.719875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732962+0000","last_change":"2023-10-07T07:57:50.728894+0000","last_active":"2023-10-07T07:57:50.732962+0000","last_peered":"2023-10-07T07:57:50.732962+0000","last_clean":"2023-10-07T07:57:50.732962+0000","last_became_active":"2023-10-07T07:57:50.726313+0000","last_became_peered":"2023-10-07T07:57:50.726313+0000","last_unstale":"2023-10-07T07:57:50.732962+0000","last_undegraded":"2023-10-07T07:57:50.732962+0000","last_fullsized":"2023-10-07T07:57:50.732962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.736796+0000","last_change":"2023-10-07T07:57:50.730368+0000","last_active":"2023-10-07T07:57:50.736796+0000","last_peered":"2023-10-07T07:57:50.736796+0000","last_clean":"2023-10-07T07:57:50.736796+0000","last_became_active":"2023-10-07T07:57:50.730094+0000","last_became_peered":"2023-10-07T07:57:50.730094+0000","last_unstale":"2023-10-07T07:57:50.736796+0000","last_undegraded":"2023-10-07T07:57:50.736796+0000","last_fullsized":"2023-10-07T07:57:50.736796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:53:49.221151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734017+0000","last_change":"2023-10-07T07:57:50.726948+0000","last_active":"2023-10-07T07:57:50.734017+0000","last_peered":"2023-10-07T07:57:50.734017+0000","last_clean":"2023-10-07T07:57:50.734017+0000","last_became_active":"2023-10-07T07:57:50.726622+0000","last_became_peered":"2023-10-07T07:57:50.726622+0000","last_unstale":"2023-10-07T07:57:50.734017+0000","last_undegraded":"2023-10-07T07:57:50.734017+0000","last_fullsized":"2023-10-07T07:57:50.734017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:38:53.980034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.745995+0000","last_change":"2023-10-07T07:57:50.745365+0000","last_active":"2023-10-07T07:57:50.745995+0000","last_peered":"2023-10-07T07:57:50.745995+0000","last_clean":"2023-10-07T07:57:50.745995+0000","last_became_active":"2023-10-07T07:57:50.745061+0000","last_became_peered":"2023-10-07T07:57:50.745061+0000","last_unstale":"2023-10-07T07:57:50.745995+0000","last_undegraded":"2023-10-07T07:57:50.745995+0000","last_fullsized":"2023-10-07T07:57:50.745995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:09:38.557090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733023+0000","last_change":"2023-10-07T07:57:50.729317+0000","last_active":"2023-10-07T07:57:50.733023+0000","last_peered":"2023-10-07T07:57:50.733023+0000","last_clean":"2023-10-07T07:57:50.733023+0000","last_became_active":"2023-10-07T07:57:50.726065+0000","last_became_peered":"2023-10-07T07:57:50.726065+0000","last_unstale":"2023-10-07T07:57:50.733023+0000","last_undegraded":"2023-10-07T07:57:50.733023+0000","last_fullsized":"2023-10-07T07:57:50.733023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:59:47.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735217+0000","last_change":"2023-10-07T07:57:50.734224+0000","last_active":"2023-10-07T07:57:50.735217+0000","last_peered":"2023-10-07T07:57:50.735217+0000","last_clean":"2023-10-07T07:57:50.735217+0000","last_became_active":"2023-10-07T07:57:50.729939+0000","last_became_peered":"2023-10-07T07:57:50.729939+0000","last_unstale":"2023-10-07T07:57:50.735217+0000","last_undegraded":"2023-10-07T07:57:50.735217+0000","last_fullsized":"2023-10-07T07:57:50.735217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:53:49.221151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735896+0000","last_change":"2023-10-07T07:57:50.730055+0000","last_active":"2023-10-07T07:57:50.735896+0000","last_peered":"2023-10-07T07:57:50.735896+0000","last_clean":"2023-10-07T07:57:50.735896+0000","last_became_active":"2023-10-07T07:57:50.729800+0000","last_became_peered":"2023-10-07T07:57:50.729800+0000","last_unstale":"2023-10-07T07:57:50.735896+0000","last_undegraded":"2023-10-07T07:57:50.735896+0000","last_fullsized":"2023-10-07T07:57:50.735896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:37:11.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725487+0000","last_change":"2023-10-07T07:57:50.718932+0000","last_active":"2023-10-07T07:57:50.725487+0000","last_peered":"2023-10-07T07:57:50.725487+0000","last_clean":"2023-10-07T07:57:50.725487+0000","last_became_active":"2023-10-07T07:57:50.718650+0000","last_became_peered":"2023-10-07T07:57:50.718650+0000","last_unstale":"2023-10-07T07:57:50.725487+0000","last_undegraded":"2023-10-07T07:57:50.725487+0000","last_fullsized":"2023-10-07T07:57:50.725487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:40.473036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.737108+0000","last_change":"2023-10-07T07:57:50.730279+0000","last_active":"2023-10-07T07:57:50.737108+0000","last_peered":"2023-10-07T07:57:50.737108+0000","last_clean":"2023-10-07T07:57:50.737108+0000","last_became_active":"2023-10-07T07:57:50.730004+0000","last_became_peered":"2023-10-07T07:57:50.730004+0000","last_unstale":"2023-10-07T07:57:50.737108+0000","last_undegraded":"2023-10-07T07:57:50.737108+0000","last_fullsized":"2023-10-07T07:57:50.737108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.720796+0000","last_change":"2023-10-07T07:57:50.720796+0000","last_active":"2023-10-07T07:57:50.720796+0000","last_peered":"2023-10-07T07:57:50.720796+0000","last_clean":"2023-10-07T07:57:50.720796+0000","last_became_active":"2023-10-07T07:57:50.720416+0000","last_became_peered":"2023-10-07T07:57:50.720416+0000","last_unstale":"2023-10-07T07:57:50.720796+0000","last_undegraded":"2023-10-07T07:57:50.720796+0000","last_fullsized":"2023-10-07T07:57:50.720796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:03:44.885267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741002+0000","last_change":"2023-10-07T07:57:50.741002+0000","last_active":"2023-10-07T07:57:50.741002+0000","last_peered":"2023-10-07T07:57:50.741002+0000","last_clean":"2023-10-07T07:57:50.741002+0000","last_became_active":"2023-10-07T07:57:50.740515+0000","last_became_peered":"2023-10-07T07:57:50.740515+0000","last_unstale":"2023-10-07T07:57:50.741002+0000","last_undegraded":"2023-10-07T07:57:50.741002+0000","last_fullsized":"2023-10-07T07:57:50.741002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:35:45.958299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.731000+0000","last_change":"2023-10-07T07:57:50.725365+0000","last_active":"2023-10-07T07:57:50.731000+0000","last_peered":"2023-10-07T07:57:50.731000+0000","last_clean":"2023-10-07T07:57:50.731000+0000","last_became_active":"2023-10-07T07:57:50.725076+0000","last_became_peered":"2023-10-07T07:57:50.725076+0000","last_unstale":"2023-10-07T07:57:50.731000+0000","last_undegraded":"2023-10-07T07:57:50.731000+0000","last_fullsized":"2023-10-07T07:57:50.731000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722362+0000","last_change":"2023-10-07T07:57:50.722362+0000","last_active":"2023-10-07T07:57:50.722362+0000","last_peered":"2023-10-07T07:57:50.722362+0000","last_clean":"2023-10-07T07:57:50.722362+0000","last_became_active":"2023-10-07T07:57:50.722022+0000","last_became_peered":"2023-10-07T07:57:50.722022+0000","last_unstale":"2023-10-07T07:57:50.722362+0000","last_undegraded":"2023-10-07T07:57:50.722362+0000","last_fullsized":"2023-10-07T07:57:50.722362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:25:31.456924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742832+0000","last_change":"2023-10-07T07:57:49.726945+0000","last_active":"2023-10-07T07:57:50.742832+0000","last_peered":"2023-10-07T07:57:50.742832+0000","last_clean":"2023-10-07T07:57:50.742832+0000","last_became_active":"2023-10-07T07:57:49.726621+0000","last_became_peered":"2023-10-07T07:57:49.726621+0000","last_unstale":"2023-10-07T07:57:50.742832+0000","last_undegraded":"2023-10-07T07:57:50.742832+0000","last_fullsized":"2023-10-07T07:57:50.742832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741522+0000","last_change":"2023-10-07T07:57:50.740835+0000","last_active":"2023-10-07T07:57:50.741522+0000","last_peered":"2023-10-07T07:57:50.741522+0000","last_clean":"2023-10-07T07:57:50.741522+0000","last_became_active":"2023-10-07T07:57:50.740562+0000","last_became_peered":"2023-10-07T07:57:50.740562+0000","last_unstale":"2023-10-07T07:57:50.741522+0000","last_undegraded":"2023-10-07T07:57:50.741522+0000","last_fullsized":"2023-10-07T07:57:50.741522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:09:38.557090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738888+0000","last_change":"2023-10-07T07:57:49.725329+0000","last_active":"2023-10-07T07:57:50.738888+0000","last_peered":"2023-10-07T07:57:50.738888+0000","last_clean":"2023-10-07T07:57:50.738888+0000","last_became_active":"2023-10-07T07:57:49.724930+0000","last_became_peered":"2023-10-07T07:57:49.724930+0000","last_unstale":"2023-10-07T07:57:50.738888+0000","last_undegraded":"2023-10-07T07:57:50.738888+0000","last_fullsized":"2023-10-07T07:57:50.738888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725494+0000","last_change":"2023-10-07T07:57:50.724035+0000","last_active":"2023-10-07T07:57:50.725494+0000","last_peered":"2023-10-07T07:57:50.725494+0000","last_clean":"2023-10-07T07:57:50.725494+0000","last_became_active":"2023-10-07T07:57:50.723418+0000","last_became_peered":"2023-10-07T07:57:50.723418+0000","last_unstale":"2023-10-07T07:57:50.725494+0000","last_undegraded":"2023-10-07T07:57:50.725494+0000","last_fullsized":"2023-10-07T07:57:50.725494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741778+0000","last_change":"2023-10-07T07:57:49.727656+0000","last_active":"2023-10-07T07:57:50.741778+0000","last_peered":"2023-10-07T07:57:50.741778+0000","last_clean":"2023-10-07T07:57:50.741778+0000","last_became_active":"2023-10-07T07:57:49.727380+0000","last_became_peered":"2023-10-07T07:57:49.727380+0000","last_unstale":"2023-10-07T07:57:50.741778+0000","last_undegraded":"2023-10-07T07:57:50.741778+0000","last_fullsized":"2023-10-07T07:57:50.741778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:31.962724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732428+0000","last_change":"2023-10-07T07:57:50.727657+0000","last_active":"2023-10-07T07:57:50.732428+0000","last_peered":"2023-10-07T07:57:50.732428+0000","last_clean":"2023-10-07T07:57:50.732428+0000","last_became_active":"2023-10-07T07:57:50.727309+0000","last_became_peered":"2023-10-07T07:57:50.727309+0000","last_unstale":"2023-10-07T07:57:50.732428+0000","last_undegraded":"2023-10-07T07:57:50.732428+0000","last_fullsized":"2023-10-07T07:57:50.732428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743139+0000","last_change":"2023-10-07T07:57:49.729636+0000","last_active":"2023-10-07T07:57:50.743139+0000","last_peered":"2023-10-07T07:57:50.743139+0000","last_clean":"2023-10-07T07:57:50.743139+0000","last_became_active":"2023-10-07T07:57:49.725076+0000","last_became_peered":"2023-10-07T07:57:49.725076+0000","last_unstale":"2023-10-07T07:57:50.743139+0000","last_undegraded":"2023-10-07T07:57:50.743139+0000","last_fullsized":"2023-10-07T07:57:50.743139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:01.309236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742189+0000","last_change":"2023-10-07T07:57:50.742189+0000","last_active":"2023-10-07T07:57:50.742189+0000","last_peered":"2023-10-07T07:57:50.742189+0000","last_clean":"2023-10-07T07:57:50.742189+0000","last_became_active":"2023-10-07T07:57:50.741823+0000","last_became_peered":"2023-10-07T07:57:50.741823+0000","last_unstale":"2023-10-07T07:57:50.742189+0000","last_undegraded":"2023-10-07T07:57:50.742189+0000","last_fullsized":"2023-10-07T07:57:50.742189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:18:23.885325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738865+0000","last_change":"2023-10-07T07:57:49.724287+0000","last_active":"2023-10-07T07:57:50.738865+0000","last_peered":"2023-10-07T07:57:50.738865+0000","last_clean":"2023-10-07T07:57:50.738865+0000","last_became_active":"2023-10-07T07:57:49.723711+0000","last_became_peered":"2023-10-07T07:57:49.723711+0000","last_unstale":"2023-10-07T07:57:50.738865+0000","last_undegraded":"2023-10-07T07:57:50.738865+0000","last_fullsized":"2023-10-07T07:57:50.738865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726639+0000","last_change":"2023-10-07T07:57:50.723737+0000","last_active":"2023-10-07T07:57:50.726639+0000","last_peered":"2023-10-07T07:57:50.726639+0000","last_clean":"2023-10-07T07:57:50.726639+0000","last_became_active":"2023-10-07T07:57:50.723444+0000","last_became_peered":"2023-10-07T07:57:50.723444+0000","last_unstale":"2023-10-07T07:57:50.726639+0000","last_undegraded":"2023-10-07T07:57:50.726639+0000","last_fullsized":"2023-10-07T07:57:50.726639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712156+0000","last_change":"2023-10-07T07:57:49.737749+0000","last_active":"2023-10-07T07:57:50.712156+0000","last_peered":"2023-10-07T07:57:50.712156+0000","last_clean":"2023-10-07T07:57:50.712156+0000","last_became_active":"2023-10-07T07:57:49.737477+0000","last_became_peered":"2023-10-07T07:57:49.737477+0000","last_unstale":"2023-10-07T07:57:50.712156+0000","last_undegraded":"2023-10-07T07:57:50.712156+0000","last_fullsized":"2023-10-07T07:57:50.712156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:10.280748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.730239+0000","last_change":"2023-10-07T07:57:50.730239+0000","last_active":"2023-10-07T07:57:50.730239+0000","last_peered":"2023-10-07T07:57:50.730239+0000","last_clean":"2023-10-07T07:57:50.730239+0000","last_became_active":"2023-10-07T07:57:50.729972+0000","last_became_peered":"2023-10-07T07:57:50.729972+0000","last_unstale":"2023-10-07T07:57:50.730239+0000","last_undegraded":"2023-10-07T07:57:50.730239+0000","last_fullsized":"2023-10-07T07:57:50.730239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:03:06.838727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711739+0000","last_change":"2023-10-07T07:57:49.728260+0000","last_active":"2023-10-07T07:57:50.711739+0000","last_peered":"2023-10-07T07:57:50.711739+0000","last_clean":"2023-10-07T07:57:50.711739+0000","last_became_active":"2023-10-07T07:57:49.728019+0000","last_became_peered":"2023-10-07T07:57:49.728019+0000","last_unstale":"2023-10-07T07:57:50.711739+0000","last_undegraded":"2023-10-07T07:57:50.711739+0000","last_fullsized":"2023-10-07T07:57:50.711739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726987+0000","last_change":"2023-10-07T07:57:50.723700+0000","last_active":"2023-10-07T07:57:50.726987+0000","last_peered":"2023-10-07T07:57:50.726987+0000","last_clean":"2023-10-07T07:57:50.726987+0000","last_became_active":"2023-10-07T07:57:50.723420+0000","last_became_peered":"2023-10-07T07:57:50.723420+0000","last_unstale":"2023-10-07T07:57:50.726987+0000","last_undegraded":"2023-10-07T07:57:50.726987+0000","last_fullsized":"2023-10-07T07:57:50.726987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:26:00.336772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.736925+0000","last_change":"2023-10-07T07:57:49.732565+0000","last_active":"2023-10-07T07:57:49.736925+0000","last_peered":"2023-10-07T07:57:49.736925+0000","last_clean":"2023-10-07T07:57:49.736925+0000","last_became_active":"2023-10-07T07:57:49.732250+0000","last_became_peered":"2023-10-07T07:57:49.732250+0000","last_unstale":"2023-10-07T07:57:49.736925+0000","last_undegraded":"2023-10-07T07:57:49.736925+0000","last_fullsized":"2023-10-07T07:57:49.736925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722227+0000","last_change":"2023-10-07T07:57:50.722227+0000","last_active":"2023-10-07T07:57:50.722227+0000","last_peered":"2023-10-07T07:57:50.722227+0000","last_clean":"2023-10-07T07:57:50.722227+0000","last_became_active":"2023-10-07T07:57:50.721953+0000","last_became_peered":"2023-10-07T07:57:50.721953+0000","last_unstale":"2023-10-07T07:57:50.722227+0000","last_undegraded":"2023-10-07T07:57:50.722227+0000","last_fullsized":"2023-10-07T07:57:50.722227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:50:56.892918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.709932+0000","last_change":"2023-10-07T07:57:49.720571+0000","last_active":"2023-10-07T07:57:50.709932+0000","last_peered":"2023-10-07T07:57:50.709932+0000","last_clean":"2023-10-07T07:57:50.709932+0000","last_became_active":"2023-10-07T07:57:49.720226+0000","last_became_peered":"2023-10-07T07:57:49.720226+0000","last_unstale":"2023-10-07T07:57:50.709932+0000","last_undegraded":"2023-10-07T07:57:50.709932+0000","last_fullsized":"2023-10-07T07:57:50.709932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724121+0000","last_change":"2023-10-07T07:57:50.724121+0000","last_active":"2023-10-07T07:57:50.724121+0000","last_peered":"2023-10-07T07:57:50.724121+0000","last_clean":"2023-10-07T07:57:50.724121+0000","last_became_active":"2023-10-07T07:57:50.723789+0000","last_became_peered":"2023-10-07T07:57:50.723789+0000","last_unstale":"2023-10-07T07:57:50.724121+0000","last_undegraded":"2023-10-07T07:57:50.724121+0000","last_fullsized":"2023-10-07T07:57:50.724121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.720435+0000","last_change":"2023-10-07T07:57:49.733414+0000","last_active":"2023-10-07T07:57:50.720435+0000","last_peered":"2023-10-07T07:57:50.720435+0000","last_clean":"2023-10-07T07:57:50.720435+0000","last_became_active":"2023-10-07T07:57:49.732736+0000","last_became_peered":"2023-10-07T07:57:49.732736+0000","last_unstale":"2023-10-07T07:57:50.720435+0000","last_undegraded":"2023-10-07T07:57:50.720435+0000","last_fullsized":"2023-10-07T07:57:50.720435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:49.692513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727348+0000","last_change":"2023-10-07T07:57:50.724743+0000","last_active":"2023-10-07T07:57:50.727348+0000","last_peered":"2023-10-07T07:57:50.727348+0000","last_clean":"2023-10-07T07:57:50.727348+0000","last_became_active":"2023-10-07T07:57:50.724501+0000","last_became_peered":"2023-10-07T07:57:50.724501+0000","last_unstale":"2023-10-07T07:57:50.727348+0000","last_undegraded":"2023-10-07T07:57:50.727348+0000","last_fullsized":"2023-10-07T07:57:50.727348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:03:23.294266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738823+0000","last_change":"2023-10-07T07:57:49.721843+0000","last_active":"2023-10-07T07:57:50.738823+0000","last_peered":"2023-10-07T07:57:50.738823+0000","last_clean":"2023-10-07T07:57:50.738823+0000","last_became_active":"2023-10-07T07:57:49.721533+0000","last_became_peered":"2023-10-07T07:57:49.721533+0000","last_unstale":"2023-10-07T07:57:50.738823+0000","last_undegraded":"2023-10-07T07:57:50.738823+0000","last_fullsized":"2023-10-07T07:57:50.738823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724096+0000","last_change":"2023-10-07T07:57:50.724096+0000","last_active":"2023-10-07T07:57:50.724096+0000","last_peered":"2023-10-07T07:57:50.724096+0000","last_clean":"2023-10-07T07:57:50.724096+0000","last_became_active":"2023-10-07T07:57:50.723782+0000","last_became_peered":"2023-10-07T07:57:50.723782+0000","last_unstale":"2023-10-07T07:57:50.724096+0000","last_undegraded":"2023-10-07T07:57:50.724096+0000","last_fullsized":"2023-10-07T07:57:50.724096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:59:47.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742091+0000","last_change":"2023-10-07T07:57:49.720185+0000","last_active":"2023-10-07T07:57:50.742091+0000","last_peered":"2023-10-07T07:57:50.742091+0000","last_clean":"2023-10-07T07:57:50.742091+0000","last_became_active":"2023-10-07T07:57:49.719880+0000","last_became_peered":"2023-10-07T07:57:49.719880+0000","last_unstale":"2023-10-07T07:57:50.742091+0000","last_undegraded":"2023-10-07T07:57:50.742091+0000","last_fullsized":"2023-10-07T07:57:50.742091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:53:25.421958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732557+0000","last_change":"2023-10-07T07:57:50.726280+0000","last_active":"2023-10-07T07:57:50.732557+0000","last_peered":"2023-10-07T07:57:50.732557+0000","last_clean":"2023-10-07T07:57:50.732557+0000","last_became_active":"2023-10-07T07:57:50.725649+0000","last_became_peered":"2023-10-07T07:57:50.725649+0000","last_unstale":"2023-10-07T07:57:50.732557+0000","last_undegraded":"2023-10-07T07:57:50.732557+0000","last_fullsized":"2023-10-07T07:57:50.732557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:22:50.873696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739735+0000","last_change":"2023-10-07T07:57:49.732331+0000","last_active":"2023-10-07T07:57:50.739735+0000","last_peered":"2023-10-07T07:57:50.739735+0000","last_clean":"2023-10-07T07:57:50.739735+0000","last_became_active":"2023-10-07T07:57:49.732085+0000","last_became_peered":"2023-10-07T07:57:49.732085+0000","last_unstale":"2023-10-07T07:57:50.739735+0000","last_undegraded":"2023-10-07T07:57:50.739735+0000","last_fullsized":"2023-10-07T07:57:50.739735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727938+0000","last_change":"2023-10-07T07:57:50.724196+0000","last_active":"2023-10-07T07:57:50.727938+0000","last_peered":"2023-10-07T07:57:50.727938+0000","last_clean":"2023-10-07T07:57:50.727938+0000","last_became_active":"2023-10-07T07:57:50.723911+0000","last_became_peered":"2023-10-07T07:57:50.723911+0000","last_unstale":"2023-10-07T07:57:50.727938+0000","last_undegraded":"2023-10-07T07:57:50.727938+0000","last_fullsized":"2023-10-07T07:57:50.727938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:42.757604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737857+0000","last_change":"2023-10-07T07:57:49.734820+0000","last_active":"2023-10-07T07:57:49.737857+0000","last_peered":"2023-10-07T07:57:49.737857+0000","last_clean":"2023-10-07T07:57:49.737857+0000","last_became_active":"2023-10-07T07:57:49.732635+0000","last_became_peered":"2023-10-07T07:57:49.732635+0000","last_unstale":"2023-10-07T07:57:49.737857+0000","last_undegraded":"2023-10-07T07:57:49.737857+0000","last_fullsized":"2023-10-07T07:57:49.737857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:36:22.098994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740602+0000","last_change":"2023-10-07T07:57:50.740602+0000","last_active":"2023-10-07T07:57:50.740602+0000","last_peered":"2023-10-07T07:57:50.740602+0000","last_clean":"2023-10-07T07:57:50.740602+0000","last_became_active":"2023-10-07T07:57:50.740273+0000","last_became_peered":"2023-10-07T07:57:50.740273+0000","last_unstale":"2023-10-07T07:57:50.740602+0000","last_undegraded":"2023-10-07T07:57:50.740602+0000","last_fullsized":"2023-10-07T07:57:50.740602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:33:31.464165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.716494+0000","last_change":"2023-10-07T07:57:49.738688+0000","last_active":"2023-10-07T07:57:50.716494+0000","last_peered":"2023-10-07T07:57:50.716494+0000","last_clean":"2023-10-07T07:57:50.716494+0000","last_became_active":"2023-10-07T07:57:49.738387+0000","last_became_peered":"2023-10-07T07:57:49.738387+0000","last_unstale":"2023-10-07T07:57:50.716494+0000","last_undegraded":"2023-10-07T07:57:50.716494+0000","last_fullsized":"2023-10-07T07:57:50.716494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726565+0000","last_change":"2023-10-07T07:57:50.723301+0000","last_active":"2023-10-07T07:57:50.726565+0000","last_peered":"2023-10-07T07:57:50.726565+0000","last_clean":"2023-10-07T07:57:50.726565+0000","last_became_active":"2023-10-07T07:57:50.722920+0000","last_became_peered":"2023-10-07T07:57:50.722920+0000","last_unstale":"2023-10-07T07:57:50.726565+0000","last_undegraded":"2023-10-07T07:57:50.726565+0000","last_fullsized":"2023-10-07T07:57:50.726565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:32.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739521+0000","last_change":"2023-10-07T07:57:49.719344+0000","last_active":"2023-10-07T07:57:50.739521+0000","last_peered":"2023-10-07T07:57:50.739521+0000","last_clean":"2023-10-07T07:57:50.739521+0000","last_became_active":"2023-10-07T07:57:49.719053+0000","last_became_peered":"2023-10-07T07:57:49.719053+0000","last_unstale":"2023-10-07T07:57:50.739521+0000","last_undegraded":"2023-10-07T07:57:50.739521+0000","last_fullsized":"2023-10-07T07:57:50.739521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:32:15.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724500+0000","last_change":"2023-10-07T07:57:50.724500+0000","last_active":"2023-10-07T07:57:50.724500+0000","last_peered":"2023-10-07T07:57:50.724500+0000","last_clean":"2023-10-07T07:57:50.724500+0000","last_became_active":"2023-10-07T07:57:50.724175+0000","last_became_peered":"2023-10-07T07:57:50.724175+0000","last_unstale":"2023-10-07T07:57:50.724500+0000","last_undegraded":"2023-10-07T07:57:50.724500+0000","last_fullsized":"2023-10-07T07:57:50.724500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739659+0000","last_change":"2023-10-07T07:57:49.733130+0000","last_active":"2023-10-07T07:57:50.739659+0000","last_peered":"2023-10-07T07:57:50.739659+0000","last_clean":"2023-10-07T07:57:50.739659+0000","last_became_active":"2023-10-07T07:57:49.732810+0000","last_became_peered":"2023-10-07T07:57:49.732810+0000","last_unstale":"2023-10-07T07:57:50.739659+0000","last_undegraded":"2023-10-07T07:57:50.739659+0000","last_fullsized":"2023-10-07T07:57:50.739659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:32:15.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.731887+0000","last_change":"2023-10-07T07:57:50.731887+0000","last_active":"2023-10-07T07:57:50.731887+0000","last_peered":"2023-10-07T07:57:50.731887+0000","last_clean":"2023-10-07T07:57:50.731887+0000","last_became_active":"2023-10-07T07:57:50.731535+0000","last_became_peered":"2023-10-07T07:57:50.731535+0000","last_unstale":"2023-10-07T07:57:50.731887+0000","last_undegraded":"2023-10-07T07:57:50.731887+0000","last_fullsized":"2023-10-07T07:57:50.731887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:40.473036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711292+0000","last_change":"2023-10-07T07:57:49.722522+0000","last_active":"2023-10-07T07:57:50.711292+0000","last_peered":"2023-10-07T07:57:50.711292+0000","last_clean":"2023-10-07T07:57:50.711292+0000","last_became_active":"2023-10-07T07:57:49.722225+0000","last_became_peered":"2023-10-07T07:57:49.722225+0000","last_unstale":"2023-10-07T07:57:50.711292+0000","last_undegraded":"2023-10-07T07:57:50.711292+0000","last_fullsized":"2023-10-07T07:57:50.711292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.717165+0000","last_change":"2023-10-07T07:57:50.717165+0000","last_active":"2023-10-07T07:57:50.717165+0000","last_peered":"2023-10-07T07:57:50.717165+0000","last_clean":"2023-10-07T07:57:50.717165+0000","last_became_active":"2023-10-07T07:57:50.716840+0000","last_became_peered":"2023-10-07T07:57:50.716840+0000","last_unstale":"2023-10-07T07:57:50.717165+0000","last_undegraded":"2023-10-07T07:57:50.717165+0000","last_fullsized":"2023-10-07T07:57:50.717165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:50.698077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737868+0000","last_change":"2023-10-07T07:57:49.735497+0000","last_active":"2023-10-07T07:57:49.737868+0000","last_peered":"2023-10-07T07:57:49.737868+0000","last_clean":"2023-10-07T07:57:49.737868+0000","last_became_active":"2023-10-07T07:57:49.735195+0000","last_became_peered":"2023-10-07T07:57:49.735195+0000","last_unstale":"2023-10-07T07:57:49.737868+0000","last_undegraded":"2023-10-07T07:57:49.737868+0000","last_fullsized":"2023-10-07T07:57:49.737868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:39.467472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742402+0000","last_change":"2023-10-07T07:57:50.741594+0000","last_active":"2023-10-07T07:57:50.742402+0000","last_peered":"2023-10-07T07:57:50.742402+0000","last_clean":"2023-10-07T07:57:50.742402+0000","last_became_active":"2023-10-07T07:57:50.741331+0000","last_became_peered":"2023-10-07T07:57:50.741331+0000","last_unstale":"2023-10-07T07:57:50.742402+0000","last_undegraded":"2023-10-07T07:57:50.742402+0000","last_fullsized":"2023-10-07T07:57:50.742402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:26:00.336772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733078+0000","last_change":"2023-10-07T07:57:50.726796+0000","last_active":"2023-10-07T07:57:50.733078+0000","last_peered":"2023-10-07T07:57:50.733078+0000","last_clean":"2023-10-07T07:57:50.733078+0000","last_became_active":"2023-10-07T07:57:50.725206+0000","last_became_peered":"2023-10-07T07:57:50.725206+0000","last_unstale":"2023-10-07T07:57:50.733078+0000","last_undegraded":"2023-10-07T07:57:50.733078+0000","last_fullsized":"2023-10-07T07:57:50.733078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:46:16.424520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.714634+0000","last_change":"2023-10-07T07:57:49.722494+0000","last_active":"2023-10-07T07:57:50.714634+0000","last_peered":"2023-10-07T07:57:50.714634+0000","last_clean":"2023-10-07T07:57:50.714634+0000","last_became_active":"2023-10-07T07:57:49.722237+0000","last_became_peered":"2023-10-07T07:57:49.722237+0000","last_unstale":"2023-10-07T07:57:50.714634+0000","last_undegraded":"2023-10-07T07:57:50.714634+0000","last_fullsized":"2023-10-07T07:57:50.714634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.718217+0000","last_change":"2023-10-07T07:57:50.718217+0000","last_active":"2023-10-07T07:57:50.718217+0000","last_peered":"2023-10-07T07:57:50.718217+0000","last_clean":"2023-10-07T07:57:50.718217+0000","last_became_active":"2023-10-07T07:57:50.717791+0000","last_became_peered":"2023-10-07T07:57:50.717791+0000","last_unstale":"2023-10-07T07:57:50.718217+0000","last_undegraded":"2023-10-07T07:57:50.718217+0000","last_fullsized":"2023-10-07T07:57:50.718217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:42.757604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737185+0000","last_change":"2023-10-07T07:57:49.733338+0000","last_active":"2023-10-07T07:57:49.737185+0000","last_peered":"2023-10-07T07:57:49.737185+0000","last_clean":"2023-10-07T07:57:49.737185+0000","last_became_active":"2023-10-07T07:57:49.732478+0000","last_became_peered":"2023-10-07T07:57:49.732478+0000","last_unstale":"2023-10-07T07:57:49.737185+0000","last_undegraded":"2023-10-07T07:57:49.737185+0000","last_fullsized":"2023-10-07T07:57:49.737185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:01.309236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.709777+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T07:57:50.709777+0000","last_peered":"2023-10-07T07:57:50.709777+0000","last_clean":"2023-10-07T07:57:50.709777+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T07:57:50.709777+0000","last_undegraded":"2023-10-07T07:57:50.709777+0000","last_fullsized":"2023-10-07T07:57:50.709777+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727905+0000","last_change":"2023-10-07T07:57:50.723683+0000","last_active":"2023-10-07T07:57:50.727905+0000","last_peered":"2023-10-07T07:57:50.727905+0000","last_clean":"2023-10-07T07:57:50.727905+0000","last_became_active":"2023-10-07T07:57:50.723408+0000","last_became_peered":"2023-10-07T07:57:50.723408+0000","last_unstale":"2023-10-07T07:57:50.727905+0000","last_undegraded":"2023-10-07T07:57:50.727905+0000","last_fullsized":"2023-10-07T07:57:50.727905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:11.286312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737533+0000","last_change":"2023-10-07T07:57:49.733672+0000","last_active":"2023-10-07T07:57:49.737533+0000","last_peered":"2023-10-07T07:57:49.737533+0000","last_clean":"2023-10-07T07:57:49.737533+0000","last_became_active":"2023-10-07T07:57:49.733375+0000","last_became_peered":"2023-10-07T07:57:49.733375+0000","last_unstale":"2023-10-07T07:57:49.737533+0000","last_undegraded":"2023-10-07T07:57:49.737533+0000","last_fullsized":"2023-10-07T07:57:49.737533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.736882+0000","last_change":"2023-10-07T07:57:49.730927+0000","last_active":"2023-10-07T07:57:49.736882+0000","last_peered":"2023-10-07T07:57:49.736882+0000","last_clean":"2023-10-07T07:57:49.736882+0000","last_became_active":"2023-10-07T07:57:49.727008+0000","last_became_peered":"2023-10-07T07:57:49.727008+0000","last_unstale":"2023-10-07T07:57:49.736882+0000","last_undegraded":"2023-10-07T07:57:49.736882+0000","last_fullsized":"2023-10-07T07:57:49.736882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727827+0000","last_change":"2023-10-07T07:57:50.723292+0000","last_active":"2023-10-07T07:57:50.727827+0000","last_peered":"2023-10-07T07:57:50.727827+0000","last_clean":"2023-10-07T07:57:50.727827+0000","last_became_active":"2023-10-07T07:57:50.722388+0000","last_became_peered":"2023-10-07T07:57:50.722388+0000","last_unstale":"2023-10-07T07:57:50.727827+0000","last_undegraded":"2023-10-07T07:57:50.727827+0000","last_fullsized":"2023-10-07T07:57:50.727827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:53:57.122070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.740394+0000","last_change":"2023-10-07T07:57:49.736910+0000","last_active":"2023-10-07T07:57:49.740394+0000","last_peered":"2023-10-07T07:57:49.740394+0000","last_clean":"2023-10-07T07:57:49.740394+0000","last_became_active":"2023-10-07T07:57:49.736645+0000","last_became_peered":"2023-10-07T07:57:49.736645+0000","last_unstale":"2023-10-07T07:57:49.740394+0000","last_undegraded":"2023-10-07T07:57:49.740394+0000","last_fullsized":"2023-10-07T07:57:49.740394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.721660+0000","last_change":"2023-10-07T07:57:50.721660+0000","last_active":"2023-10-07T07:57:50.721660+0000","last_peered":"2023-10-07T07:57:50.721660+0000","last_clean":"2023-10-07T07:57:50.721660+0000","last_became_active":"2023-10-07T07:57:50.721390+0000","last_became_peered":"2023-10-07T07:57:50.721390+0000","last_unstale":"2023-10-07T07:57:50.721660+0000","last_undegraded":"2023-10-07T07:57:50.721660+0000","last_fullsized":"2023-10-07T07:57:50.721660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:11.286312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.745147+0000","last_change":"2023-10-07T07:57:49.737869+0000","last_active":"2023-10-07T07:57:50.745147+0000","last_peered":"2023-10-07T07:57:50.745147+0000","last_clean":"2023-10-07T07:57:50.745147+0000","last_became_active":"2023-10-07T07:57:49.737571+0000","last_became_peered":"2023-10-07T07:57:49.737571+0000","last_unstale":"2023-10-07T07:57:50.745147+0000","last_undegraded":"2023-10-07T07:57:50.745147+0000","last_fullsized":"2023-10-07T07:57:50.745147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:49.692513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.718352+0000","last_change":"2023-10-07T07:57:50.718352+0000","last_active":"2023-10-07T07:57:50.718352+0000","last_peered":"2023-10-07T07:57:50.718352+0000","last_clean":"2023-10-07T07:57:50.718352+0000","last_became_active":"2023-10-07T07:57:50.718091+0000","last_became_peered":"2023-10-07T07:57:50.718091+0000","last_unstale":"2023-10-07T07:57:50.718352+0000","last_undegraded":"2023-10-07T07:57:50.718352+0000","last_fullsized":"2023-10-07T07:57:50.718352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:37:11.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.714964+0000","last_change":"2023-10-07T07:57:49.730601+0000","last_active":"2023-10-07T07:57:50.714964+0000","last_peered":"2023-10-07T07:57:50.714964+0000","last_clean":"2023-10-07T07:57:50.714964+0000","last_became_active":"2023-10-07T07:57:49.730370+0000","last_became_peered":"2023-10-07T07:57:49.730370+0000","last_unstale":"2023-10-07T07:57:50.714964+0000","last_undegraded":"2023-10-07T07:57:50.714964+0000","last_fullsized":"2023-10-07T07:57:50.714964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.723162+0000","last_change":"2023-10-07T07:57:50.723162+0000","last_active":"2023-10-07T07:57:50.723162+0000","last_peered":"2023-10-07T07:57:50.723162+0000","last_clean":"2023-10-07T07:57:50.723162+0000","last_became_active":"2023-10-07T07:57:50.722848+0000","last_became_peered":"2023-10-07T07:57:50.722848+0000","last_unstale":"2023-10-07T07:57:50.723162+0000","last_undegraded":"2023-10-07T07:57:50.723162+0000","last_fullsized":"2023-10-07T07:57:50.723162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T07:57:49.737608+0000","last_change":"2023-10-07T07:57:49.731859+0000","last_active":"2023-10-07T07:57:49.737608+0000","last_peered":"2023-10-07T07:57:49.737608+0000","last_clean":"2023-10-07T07:57:49.737608+0000","last_became_active":"2023-10-07T07:57:49.731613+0000","last_became_peered":"2023-10-07T07:57:49.731613+0000","last_unstale":"2023-10-07T07:57:49.737608+0000","last_undegraded":"2023-10-07T07:57:49.737608+0000","last_fullsized":"2023-10-07T07:57:49.737608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.723167+0000","last_change":"2023-10-07T07:57:50.723167+0000","last_active":"2023-10-07T07:57:50.723167+0000","last_peered":"2023-10-07T07:57:50.723167+0000","last_clean":"2023-10-07T07:57:50.723167+0000","last_became_active":"2023-10-07T07:57:50.722863+0000","last_became_peered":"2023-10-07T07:57:50.722863+0000","last_unstale":"2023-10-07T07:57:50.723167+0000","last_undegraded":"2023-10-07T07:57:50.723167+0000","last_fullsized":"2023-10-07T07:57:50.723167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711876+0000","last_change":"2023-10-07T07:57:49.725087+0000","last_active":"2023-10-07T07:57:50.711876+0000","last_peered":"2023-10-07T07:57:50.711876+0000","last_clean":"2023-10-07T07:57:50.711876+0000","last_became_active":"2023-10-07T07:57:49.724027+0000","last_became_peered":"2023-10-07T07:57:49.724027+0000","last_unstale":"2023-10-07T07:57:50.711876+0000","last_undegraded":"2023-10-07T07:57:50.711876+0000","last_fullsized":"2023-10-07T07:57:50.711876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722299+0000","last_change":"2023-10-07T07:57:50.717348+0000","last_active":"2023-10-07T07:57:50.722299+0000","last_peered":"2023-10-07T07:57:50.722299+0000","last_clean":"2023-10-07T07:57:50.722299+0000","last_became_active":"2023-10-07T07:57:50.716959+0000","last_became_peered":"2023-10-07T07:57:50.716959+0000","last_unstale":"2023-10-07T07:57:50.722299+0000","last_undegraded":"2023-10-07T07:57:50.722299+0000","last_fullsized":"2023-10-07T07:57:50.722299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742034+0000","last_change":"2023-10-07T07:57:49.732599+0000","last_active":"2023-10-07T07:57:50.742034+0000","last_peered":"2023-10-07T07:57:50.742034+0000","last_clean":"2023-10-07T07:57:50.742034+0000","last_became_active":"2023-10-07T07:57:49.732088+0000","last_became_peered":"2023-10-07T07:57:49.732088+0000","last_unstale":"2023-10-07T07:57:50.742034+0000","last_undegraded":"2023-10-07T07:57:50.742034+0000","last_fullsized":"2023-10-07T07:57:50.742034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:31.103335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725007+0000","last_change":"2023-10-07T07:57:50.725007+0000","last_active":"2023-10-07T07:57:50.725007+0000","last_peered":"2023-10-07T07:57:50.725007+0000","last_clean":"2023-10-07T07:57:50.725007+0000","last_became_active":"2023-10-07T07:57:50.723895+0000","last_became_peered":"2023-10-07T07:57:50.723895+0000","last_unstale":"2023-10-07T07:57:50.725007+0000","last_undegraded":"2023-10-07T07:57:50.725007+0000","last_fullsized":"2023-10-07T07:57:50.725007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:13:37.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712397+0000","last_change":"2023-10-07T07:57:49.725916+0000","last_active":"2023-10-07T07:57:50.712397+0000","last_peered":"2023-10-07T07:57:50.712397+0000","last_clean":"2023-10-07T07:57:50.712397+0000","last_became_active":"2023-10-07T07:57:49.725648+0000","last_became_peered":"2023-10-07T07:57:49.725648+0000","last_unstale":"2023-10-07T07:57:50.712397+0000","last_undegraded":"2023-10-07T07:57:50.712397+0000","last_fullsized":"2023-10-07T07:57:50.712397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.720453+0000","last_change":"2023-10-07T07:57:50.720453+0000","last_active":"2023-10-07T07:57:50.720453+0000","last_peered":"2023-10-07T07:57:50.720453+0000","last_clean":"2023-10-07T07:57:50.720453+0000","last_became_active":"2023-10-07T07:57:50.719863+0000","last_became_peered":"2023-10-07T07:57:50.719863+0000","last_unstale":"2023-10-07T07:57:50.720453+0000","last_undegraded":"2023-10-07T07:57:50.720453+0000","last_fullsized":"2023-10-07T07:57:50.720453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:21:30.452139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739318+0000","last_change":"2023-10-07T07:57:49.729606+0000","last_active":"2023-10-07T07:57:50.739318+0000","last_peered":"2023-10-07T07:57:50.739318+0000","last_clean":"2023-10-07T07:57:50.739318+0000","last_became_active":"2023-10-07T07:57:49.729372+0000","last_became_peered":"2023-10-07T07:57:49.729372+0000","last_unstale":"2023-10-07T07:57:50.739318+0000","last_undegraded":"2023-10-07T07:57:50.739318+0000","last_fullsized":"2023-10-07T07:57:50.739318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735545+0000","last_change":"2023-10-07T07:57:50.734705+0000","last_active":"2023-10-07T07:57:50.735545+0000","last_peered":"2023-10-07T07:57:50.735545+0000","last_clean":"2023-10-07T07:57:50.735545+0000","last_became_active":"2023-10-07T07:57:50.733706+0000","last_became_peered":"2023-10-07T07:57:50.733706+0000","last_unstale":"2023-10-07T07:57:50.735545+0000","last_undegraded":"2023-10-07T07:57:50.735545+0000","last_fullsized":"2023-10-07T07:57:50.735545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:56:34.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739077+0000","last_change":"2023-10-07T07:57:49.724649+0000","last_active":"2023-10-07T07:57:50.739077+0000","last_peered":"2023-10-07T07:57:50.739077+0000","last_clean":"2023-10-07T07:57:50.739077+0000","last_became_active":"2023-10-07T07:57:49.724368+0000","last_became_peered":"2023-10-07T07:57:49.724368+0000","last_unstale":"2023-10-07T07:57:50.739077+0000","last_undegraded":"2023-10-07T07:57:50.739077+0000","last_fullsized":"2023-10-07T07:57:50.739077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733770+0000","last_change":"2023-10-07T07:57:50.730366+0000","last_active":"2023-10-07T07:57:50.733770+0000","last_peered":"2023-10-07T07:57:50.733770+0000","last_clean":"2023-10-07T07:57:50.733770+0000","last_became_active":"2023-10-07T07:57:50.730012+0000","last_became_peered":"2023-10-07T07:57:50.730012+0000","last_unstale":"2023-10-07T07:57:50.733770+0000","last_undegraded":"2023-10-07T07:57:50.733770+0000","last_fullsized":"2023-10-07T07:57:50.733770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:15:16.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.721978+0000","last_change":"2023-10-07T07:57:50.721978+0000","last_active":"2023-10-07T07:57:50.721978+0000","last_peered":"2023-10-07T07:57:50.721978+0000","last_clean":"2023-10-07T07:57:50.721978+0000","last_became_active":"2023-10-07T07:57:50.719507+0000","last_became_peered":"2023-10-07T07:57:50.719507+0000","last_unstale":"2023-10-07T07:57:50.721978+0000","last_undegraded":"2023-10-07T07:57:50.721978+0000","last_fullsized":"2023-10-07T07:57:50.721978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:32.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.730933+0000","last_change":"2023-10-07T07:57:50.724692+0000","last_active":"2023-10-07T07:57:50.730933+0000","last_peered":"2023-10-07T07:57:50.730933+0000","last_clean":"2023-10-07T07:57:50.730933+0000","last_became_active":"2023-10-07T07:57:50.724411+0000","last_became_peered":"2023-10-07T07:57:50.724411+0000","last_unstale":"2023-10-07T07:57:50.730933+0000","last_undegraded":"2023-10-07T07:57:50.730933+0000","last_fullsized":"2023-10-07T07:57:50.730933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727607+0000","last_change":"2023-10-07T07:57:50.727607+0000","last_active":"2023-10-07T07:57:50.727607+0000","last_peered":"2023-10-07T07:57:50.727607+0000","last_clean":"2023-10-07T07:57:50.727607+0000","last_became_active":"2023-10-07T07:57:50.724048+0000","last_became_peered":"2023-10-07T07:57:50.724048+0000","last_unstale":"2023-10-07T07:57:50.727607+0000","last_undegraded":"2023-10-07T07:57:50.727607+0000","last_fullsized":"2023-10-07T07:57:50.727607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:38:14.739161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734147+0000","last_change":"2023-10-07T07:57:50.726261+0000","last_active":"2023-10-07T07:57:50.734147+0000","last_peered":"2023-10-07T07:57:50.734147+0000","last_clean":"2023-10-07T07:57:50.734147+0000","last_became_active":"2023-10-07T07:57:50.725636+0000","last_became_peered":"2023-10-07T07:57:50.725636+0000","last_unstale":"2023-10-07T07:57:50.734147+0000","last_undegraded":"2023-10-07T07:57:50.734147+0000","last_fullsized":"2023-10-07T07:57:50.734147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:56:34.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738656+0000","last_change":"2023-10-07T07:57:50.735497+0000","last_active":"2023-10-07T07:57:50.738656+0000","last_peered":"2023-10-07T07:57:50.738656+0000","last_clean":"2023-10-07T07:57:50.738656+0000","last_became_active":"2023-10-07T07:57:50.735172+0000","last_became_peered":"2023-10-07T07:57:50.735172+0000","last_unstale":"2023-10-07T07:57:50.738656+0000","last_undegraded":"2023-10-07T07:57:50.738656+0000","last_fullsized":"2023-10-07T07:57:50.738656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725429+0000","last_change":"2023-10-07T07:57:50.718981+0000","last_active":"2023-10-07T07:57:50.725429+0000","last_peered":"2023-10-07T07:57:50.725429+0000","last_clean":"2023-10-07T07:57:50.725429+0000","last_became_active":"2023-10-07T07:57:50.718699+0000","last_became_peered":"2023-10-07T07:57:50.718699+0000","last_unstale":"2023-10-07T07:57:50.725429+0000","last_undegraded":"2023-10-07T07:57:50.725429+0000","last_fullsized":"2023-10-07T07:57:50.725429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:36:23.104558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724708+0000","last_change":"2023-10-07T07:57:50.724708+0000","last_active":"2023-10-07T07:57:50.724708+0000","last_peered":"2023-10-07T07:57:50.724708+0000","last_clean":"2023-10-07T07:57:50.724708+0000","last_became_active":"2023-10-07T07:57:50.724343+0000","last_became_peered":"2023-10-07T07:57:50.724343+0000","last_unstale":"2023-10-07T07:57:50.724708+0000","last_undegraded":"2023-10-07T07:57:50.724708+0000","last_fullsized":"2023-10-07T07:57:50.724708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:38:53.980034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725505+0000","last_change":"2023-10-07T07:57:50.724233+0000","last_active":"2023-10-07T07:57:50.725505+0000","last_peered":"2023-10-07T07:57:50.725505+0000","last_clean":"2023-10-07T07:57:50.725505+0000","last_became_active":"2023-10-07T07:57:50.723770+0000","last_became_peered":"2023-10-07T07:57:50.723770+0000","last_unstale":"2023-10-07T07:57:50.725505+0000","last_undegraded":"2023-10-07T07:57:50.725505+0000","last_fullsized":"2023-10-07T07:57:50.725505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:22:50.873696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724839+0000","last_change":"2023-10-07T07:57:50.724839+0000","last_active":"2023-10-07T07:57:50.724839+0000","last_peered":"2023-10-07T07:57:50.724839+0000","last_clean":"2023-10-07T07:57:50.724839+0000","last_became_active":"2023-10-07T07:57:50.724489+0000","last_became_peered":"2023-10-07T07:57:50.724489+0000","last_unstale":"2023-10-07T07:57:50.724839+0000","last_undegraded":"2023-10-07T07:57:50.724839+0000","last_fullsized":"2023-10-07T07:57:50.724839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:38:11.263837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741272+0000","last_change":"2023-10-07T07:57:50.740752+0000","last_active":"2023-10-07T07:57:50.741272+0000","last_peered":"2023-10-07T07:57:50.741272+0000","last_clean":"2023-10-07T07:57:50.741272+0000","last_became_active":"2023-10-07T07:57:50.740464+0000","last_became_peered":"2023-10-07T07:57:50.740464+0000","last_unstale":"2023-10-07T07:57:50.741272+0000","last_undegraded":"2023-10-07T07:57:50.741272+0000","last_fullsized":"2023-10-07T07:57:50.741272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733824+0000","last_change":"2023-10-07T07:57:50.729375+0000","last_active":"2023-10-07T07:57:50.733824+0000","last_peered":"2023-10-07T07:57:50.733824+0000","last_clean":"2023-10-07T07:57:50.733824+0000","last_became_active":"2023-10-07T07:57:50.729038+0000","last_became_peered":"2023-10-07T07:57:50.729038+0000","last_unstale":"2023-10-07T07:57:50.733824+0000","last_undegraded":"2023-10-07T07:57:50.733824+0000","last_fullsized":"2023-10-07T07:57:50.733824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.718717+0000","last_change":"2023-10-07T07:57:50.718717+0000","last_active":"2023-10-07T07:57:50.718717+0000","last_peered":"2023-10-07T07:57:50.718717+0000","last_clean":"2023-10-07T07:57:50.718717+0000","last_became_active":"2023-10-07T07:57:50.718457+0000","last_became_peered":"2023-10-07T07:57:50.718457+0000","last_unstale":"2023-10-07T07:57:50.718717+0000","last_undegraded":"2023-10-07T07:57:50.718717+0000","last_fullsized":"2023-10-07T07:57:50.718717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:53:57.122070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735843+0000","last_change":"2023-10-07T07:57:50.734975+0000","last_active":"2023-10-07T07:57:50.735843+0000","last_peered":"2023-10-07T07:57:50.735843+0000","last_clean":"2023-10-07T07:57:50.735843+0000","last_became_active":"2023-10-07T07:57:50.734441+0000","last_became_peered":"2023-10-07T07:57:50.734441+0000","last_unstale":"2023-10-07T07:57:50.735843+0000","last_undegraded":"2023-10-07T07:57:50.735843+0000","last_fullsized":"2023-10-07T07:57:50.735843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":659463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":69081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":69579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":71205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298088,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073752,"statfs":{"total":96636764160,"available":96331522048,"internally_reserved":0,"allocated":819200,"data_stored":665773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T07:57:55.301 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-07T07:57:55.301 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-07T07:57:55.302 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-07T07:57:55.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-07T07:57:55.302 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-07T07:57:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.468+0000 7ffa39c30700 1 Processor -- start 2023-10-07T07:57:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.468+0000 7ffa39c30700 1 -- start start 2023-10-07T07:57:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa39c30700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffa3406e9d0 0x7ffa340c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa39c30700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa341562f0 0x7ffa3406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa39c30700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa3406b700 con 0x7ffa34155f20 2023-10-07T07:57:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa39c30700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa3406aaf0 con 0x7ffa341562f0 2023-10-07T07:57:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa39c30700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa3406add0 con 0x7ffa3406e9d0 2023-10-07T07:57:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa33fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa341562f0 0x7ffa3406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa33fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa341562f0 0x7ffa3406a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33554/0 (socket says 172.21.15.7:33554) 2023-10-07T07:57:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa33fff700 1 -- 172.21.15.7:0/2583960509 learned_addr learned my addr 172.21.15.7:0/2583960509 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/2583960509 >> v2:172.21.15.172:3300/0 conn(0x7ffa3406e9d0 0x7ffa340c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa33fff700 1 -- 172.21.15.7:0/2583960509 >> v2:172.21.15.172:3300/0 conn(0x7ffa3406e9d0 msgr2=0x7ffa340c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa33fff700 1 --2- 172.21.15.7:0/2583960509 >> v2:172.21.15.172:3300/0 conn(0x7ffa3406e9d0 0x7ffa340c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa33fff700 1 -- 172.21.15.7:0/2583960509 >> v1:172.21.15.172:6789/0 conn(0x7ffa34155f20 legacy=0x7ffa3406e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.469+0000 7ffa33fff700 1 -- 172.21.15.7:0/2583960509 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3406b0b0 con 0x7ffa341562f0 2023-10-07T07:57:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa33fff700 1 --2- 172.21.15.7:0/2583960509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa341562f0 0x7ffa3406a3f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa3406e0b0 tx=0x7ffa2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65aa4bb988258f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa327fc700 1 -- 172.21.15.7:0/2583960509 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00e070 con 0x7ffa341562f0 2023-10-07T07:57:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa327fc700 1 -- 172.21.15.7:0/2583960509 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c00ba20 con 0x7ffa341562f0 2023-10-07T07:57:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa327fc700 1 -- 172.21.15.7:0/2583960509 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c0092e0 con 0x7ffa341562f0 2023-10-07T07:57:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa39c30700 1 -- 172.21.15.7:0/2583960509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa341562f0 msgr2=0x7ffa3406a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa39c30700 1 --2- 172.21.15.7:0/2583960509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa341562f0 0x7ffa3406a3f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa3406e0b0 tx=0x7ffa2c009840 comp rx=0 tx=0).stop 2023-10-07T07:57:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa39c30700 1 -- 172.21.15.7:0/2583960509 shutdown_connections 2023-10-07T07:57:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa39c30700 1 --2- 172.21.15.7:0/2583960509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa341562f0 0x7ffa3406a3f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa39c30700 1 --2- 172.21.15.7:0/2583960509 >> v2:172.21.15.172:3300/0 conn(0x7ffa3406e9d0 0x7ffa340c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa39c30700 1 -- 172.21.15.7:0/2583960509 >> 172.21.15.7:0/2583960509 conn(0x7ffa34068bd0 msgr2=0x7ffa340ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa39c30700 1 -- 172.21.15.7:0/2583960509 shutdown_connections 2023-10-07T07:57:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.470+0000 7ffa39c30700 1 -- 172.21.15.7:0/2583960509 wait complete. 2023-10-07T07:57:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.472+0000 7ffa39c30700 1 Processor -- start 2023-10-07T07:57:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.472+0000 7ffa39c30700 1 -- start start 2023-10-07T07:57:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa39c30700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3406e9d0 0x7ffa340c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa39c30700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa34155f20 0x7ffa341449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa39c30700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa341562f0 0x7ffa34157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa39c30700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340c5bd0 con 0x7ffa3406e9d0 2023-10-07T07:57:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa39c30700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa341671e0 con 0x7ffa34155f20 2023-10-07T07:57:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa39c30700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340c6380 con 0x7ffa341562f0 2023-10-07T07:57:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3406e9d0 0x7ffa340c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3406e9d0 0x7ffa340c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33556/0 (socket says 172.21.15.7:33556) 2023-10-07T07:57:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa337fe700 1 -- 172.21.15.7:0/781066755 learned_addr learned my addr 172.21.15.7:0/781066755 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa33fff700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa341562f0 0x7ffa34157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa34155f20 0x7ffa341449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa337fe700 1 -- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa341562f0 msgr2=0x7ffa34157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa337fe700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa341562f0 0x7ffa34157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa337fe700 1 -- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa34155f20 msgr2=0x7ffa341449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa337fe700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa34155f20 0x7ffa341449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa337fe700 1 -- 172.21.15.7:0/781066755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c009ae0 con 0x7ffa3406e9d0 2023-10-07T07:57:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa34155f20 0x7ffa341449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:57:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa337fe700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3406e9d0 0x7ffa340c5e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c590 tx=0x7ffa2400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa1b7fe700 1 -- 172.21.15.7:0/781066755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24011590 con 0x7ffa3406e9d0 2023-10-07T07:57:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa1b7fe700 1 -- 172.21.15.7:0/781066755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa240116f0 con 0x7ffa3406e9d0 2023-10-07T07:57:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa1b7fe700 1 -- 172.21.15.7:0/781066755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24011ab0 con 0x7ffa3406e9d0 2023-10-07T07:57:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa34152780 con 0x7ffa3406e9d0 2023-10-07T07:57:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.473+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa34152d40 con 0x7ffa3406e9d0 2023-10-07T07:57:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.474+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa340716c0 con 0x7ffa3406e9d0 2023-10-07T07:57:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.475+0000 7ffa1b7fe700 1 -- 172.21.15.7:0/781066755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa2401b3b0 con 0x7ffa3406e9d0 2023-10-07T07:57:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.475+0000 7ffa1b7fe700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffa20066f30 0x7ffa200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.475+0000 7ffa1b7fe700 1 -- 172.21.15.7:0/781066755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffa24083a90 con 0x7ffa3406e9d0 2023-10-07T07:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.475+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffa20066f30 0x7ffa200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.476+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffa20066f30 0x7ffa200693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa28006f00 tx=0x7ffa28008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.480+0000 7ffa1b7fe700 1 -- 172.21.15.7:0/781066755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa24050d00 con 0x7ffa3406e9d0 2023-10-07T07:57:55.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.742+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ffa3406f1f0 con 0x7ffa3406e9d0 2023-10-07T07:57:55.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.745+0000 7ffa1b7fe700 1 -- 172.21.15.7:0/781066755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163709 (secure 0 0 0) 0x7ffa24017020 con 0x7ffa3406e9d0 2023-10-07T07:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:55.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffa20066f30 msgr2=0x7ffa200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffa20066f30 0x7ffa200693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa28006f00 tx=0x7ffa28008040 comp rx=0 tx=0).stop 2023-10-07T07:57:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3406e9d0 msgr2=0x7ffa340c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3406e9d0 0x7ffa340c5e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c590 tx=0x7ffa2400c950 comp rx=0 tx=0).stop 2023-10-07T07:57:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 shutdown_connections 2023-10-07T07:57:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffa20066f30 0x7ffa200693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa341562f0 0x7ffa34157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3406e9d0 0x7ffa340c5e40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 --2- 172.21.15.7:0/781066755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa34155f20 0x7ffa341449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 >> 172.21.15.7:0/781066755 conn(0x7ffa34068bd0 msgr2=0x7ffa340b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 shutdown_connections 2023-10-07T07:57:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.748+0000 7ffa39c30700 1 -- 172.21.15.7:0/781066755 wait complete. 2023-10-07T07:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":119541},{"type":"v1","addr":"172.21.15.172:6833","nonce":119541}]},"active_addr":"172.21.15.172:6833/119541","active_change":"2023-10-07T07:57:38.017805+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":2298359382}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":2583781638}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":1701205129}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":3241108726}]}]} 2023-10-07T07:57:55.767 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-07T07:57:55.767 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-07T07:57:55.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.897+0000 7f7c006f6700 1 Processor -- start 2023-10-07T07:57:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.897+0000 7f7c006f6700 1 -- start start 2023-10-07T07:57:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.898+0000 7f7c006f6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7bf81592c0 0x7f7bf806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.898+0000 7f7c006f6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf8159690 0x7f7bf806a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.898+0000 7f7c006f6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf806bb70 con 0x7f7bf80cab80 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.898+0000 7f7c006f6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf80cda10 con 0x7f7bf81592c0 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.898+0000 7f7c006f6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf806af60 con 0x7f7bf8159690 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfe492700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7bf81592c0 0x7f7bf806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfec93700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7bf80cab80 0x7f7bf80cafe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35938/0 (socket says 172.21.15.7:35938) 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfec93700 1 -- 172.21.15.7:0/1219632057 learned_addr learned my addr 172.21.15.7:0/1219632057 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfdc91700 1 --2- 172.21.15.7:0/1219632057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf8159690 0x7f7bf806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfe492700 1 -- 172.21.15.7:0/1219632057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf8159690 msgr2=0x7f7bf806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfe492700 1 --2- 172.21.15.7:0/1219632057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf8159690 0x7f7bf806a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfe492700 1 -- 172.21.15.7:0/1219632057 >> v1:172.21.15.7:6789/0 conn(0x7f7bf80cab80 legacy=0x7f7bf80cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfd490700 1 -- 172.21.15.7:0/1219632057 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248658279 0 0) 0x7f7bf806bb70 con 0x7f7bf80cab80 2023-10-07T07:57:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfe492700 1 -- 172.21.15.7:0/1219632057 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf806b240 con 0x7f7bf81592c0 2023-10-07T07:57:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfe492700 1 --2- 172.21.15.7:0/1219632057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf81592c0 0x7f7bf806a3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0098e0 tx=0x7f7bec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaef1c8bc4bd65f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfd490700 1 -- 172.21.15.7:0/1219632057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec009e80 con 0x7f7bf81592c0 2023-10-07T07:57:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfd490700 1 -- 172.21.15.7:0/1219632057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec00b860 con 0x7f7bf81592c0 2023-10-07T07:57:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.899+0000 7f7bfd490700 1 -- 172.21.15.7:0/1219632057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec00bbe0 con 0x7f7bf81592c0 2023-10-07T07:57:55.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.900+0000 7f7c006f6700 1 -- 172.21.15.7:0/1219632057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf81592c0 msgr2=0x7f7bf806a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.900+0000 7f7c006f6700 1 --2- 172.21.15.7:0/1219632057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf81592c0 0x7f7bf806a3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0098e0 tx=0x7f7bec0091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.900+0000 7f7c006f6700 1 -- 172.21.15.7:0/1219632057 shutdown_connections 2023-10-07T07:57:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.900+0000 7f7c006f6700 1 --2- 172.21.15.7:0/1219632057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf81592c0 0x7f7bf806a3f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.900+0000 7f7c006f6700 1 --2- 172.21.15.7:0/1219632057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf8159690 0x7f7bf806a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.900+0000 7f7c006f6700 1 -- 172.21.15.7:0/1219632057 >> 172.21.15.7:0/1219632057 conn(0x7f7bf8068bd0 msgr2=0x7f7bf80ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.900+0000 7f7c006f6700 1 -- 172.21.15.7:0/1219632057 shutdown_connections 2023-10-07T07:57:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.900+0000 7f7c006f6700 1 -- 172.21.15.7:0/1219632057 wait complete. 2023-10-07T07:57:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.901+0000 7f7c006f6700 1 Processor -- start 2023-10-07T07:57:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.901+0000 7f7c006f6700 1 -- start start 2023-10-07T07:57:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7c006f6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf80cab80 0x7f7bf8154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7c006f6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf81592c0 0x7f7bf815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7c006f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf8159690 0x7f7bf80b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7c006f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf815dda0 con 0x7f7bf8159690 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7c006f6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf815fa80 con 0x7f7bf80cab80 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7c006f6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf815d190 con 0x7f7bf81592c0 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7bfec93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf8159690 0x7f7bf80b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7bfec93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf8159690 0x7f7bf80b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33572/0 (socket says 172.21.15.7:33572) 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7bfec93700 1 -- 172.21.15.7:0/3984106577 learned_addr learned my addr 172.21.15.7:0/3984106577 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7bfe492700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf80cab80 0x7f7bf8154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.902+0000 7f7bfdc91700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf81592c0 0x7f7bf815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7bfec93700 1 -- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf81592c0 msgr2=0x7f7bf815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7bfec93700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf81592c0 0x7f7bf815c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7bfec93700 1 -- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf80cab80 msgr2=0x7f7bf8154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7bfec93700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf80cab80 0x7f7bf8154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7bfec93700 1 -- 172.21.15.7:0/3984106577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec009420 con 0x7f7bf8159690 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7bfec93700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf8159690 0x7f7bf80b1bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000cbf0 tx=0x7f7bf00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7bea7fc700 1 -- 172.21.15.7:0/3984106577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf000a8b0 con 0x7f7bf8159690 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7bea7fc700 1 -- 172.21.15.7:0/3984106577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf000aa10 con 0x7f7bf8159690 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7bea7fc700 1 -- 172.21.15.7:0/3984106577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf000add0 con 0x7f7bf8159690 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf815d470 con 0x7f7bf8159690 2023-10-07T07:57:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.903+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf815da30 con 0x7f7bf8159690 2023-10-07T07:57:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.904+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf815df50 con 0x7f7bf8159690 2023-10-07T07:57:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.904+0000 7f7bea7fc700 1 -- 172.21.15.7:0/3984106577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bf0007300 con 0x7f7bf8159690 2023-10-07T07:57:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.905+0000 7f7bea7fc700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bdc066f30 0x7f7bdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.905+0000 7f7bea7fc700 1 -- 172.21.15.7:0/3984106577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7bf0084870 con 0x7f7bf8159690 2023-10-07T07:57:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.905+0000 7f7bfe492700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bdc066f30 0x7f7bdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.906+0000 7f7bfe492700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bdc066f30 0x7f7bdc0693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0091e0 tx=0x7f7bec017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:55.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:55.910+0000 7f7bea7fc700 1 -- 172.21.15.7:0/3984106577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bf815df50 con 0x7f7bf8159690 2023-10-07T07:57:56.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.114+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bf80716c0 con 0x7f7bf8159690 2023-10-07T07:57:56.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.115+0000 7f7bea7fc700 1 -- 172.21.15.7:0/3984106577 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7bf0051ac0 con 0x7f7bf8159690 2023-10-07T07:57:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:56.116 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bdc066f30 msgr2=0x7f7bdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bdc066f30 0x7f7bdc0693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0091e0 tx=0x7f7bec017000 comp rx=0 tx=0).stop 2023-10-07T07:57:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf8159690 msgr2=0x7f7bf80b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf8159690 0x7f7bf80b1bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000cbf0 tx=0x7f7bf00019b0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 shutdown_connections 2023-10-07T07:57:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bdc066f30 0x7f7bdc0693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bf8159690 0x7f7bf80b1bd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf80cab80 0x7f7bf8154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 --2- 172.21.15.7:0/3984106577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf81592c0 0x7f7bf815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 >> 172.21.15.7:0/3984106577 conn(0x7f7bf8068bd0 msgr2=0x7f7bf806eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 shutdown_connections 2023-10-07T07:57:56.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.118+0000 7f7c006f6700 1 -- 172.21.15.7:0/3984106577 wait complete. 2023-10-07T07:57:56.133 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-07T07:57:56.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:57:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f7d521700 1 Processor -- start 2023-10-07T07:57:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f7d521700 1 -- start start 2023-10-07T07:57:56.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f7d521700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f78159380 0x7f7f780b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f7d521700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7f780d0250 0x7f7f780d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f7d521700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f780d0be0 con 0x7f7f780b3b60 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f7d521700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f780b4820 con 0x7f7f78159380 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f7d521700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f780b5000 con 0x7f7f780d0250 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f76ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f78159380 0x7f7f780b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f777fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7f780b3b60 0x7f7f780b3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35946/0 (socket says 172.21.15.7:35946) 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f777fe700 1 -- 172.21.15.7:0/386642641 learned_addr learned my addr 172.21.15.7:0/386642641 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f767fc700 1 --2- 172.21.15.7:0/386642641 >> v2:172.21.15.172:3300/0 conn(0x7f7f780d0250 0x7f7f780d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f76ffd700 1 -- 172.21.15.7:0/386642641 >> v2:172.21.15.172:3300/0 conn(0x7f7f780d0250 msgr2=0x7f7f780d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f76ffd700 1 --2- 172.21.15.7:0/386642641 >> v2:172.21.15.172:3300/0 conn(0x7f7f780d0250 0x7f7f780d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f76ffd700 1 -- 172.21.15.7:0/386642641 >> v1:172.21.15.7:6789/0 conn(0x7f7f780b3b60 legacy=0x7f7f780b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f76ffd700 1 -- 172.21.15.7:0/386642641 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f780b52e0 con 0x7f7f78159380 2023-10-07T07:57:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f75ffb700 1 -- 172.21.15.7:0/386642641 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703161480 0 0) 0x7f7f780d0be0 con 0x7f7f780b3b60 2023-10-07T07:57:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f76ffd700 1 --2- 172.21.15.7:0/386642641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 0x7f7f780b5ae0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f78067900 tx=0x7f7f680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2c9dbf39a26ca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f75ffb700 1 -- 172.21.15.7:0/386642641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68011630 con 0x7f7f78159380 2023-10-07T07:57:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f75ffb700 1 -- 172.21.15.7:0/386642641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f68011790 con 0x7f7f78159380 2023-10-07T07:57:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.266+0000 7f7f75ffb700 1 -- 172.21.15.7:0/386642641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68011b30 con 0x7f7f78159380 2023-10-07T07:57:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.267+0000 7f7f7d521700 1 -- 172.21.15.7:0/386642641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 msgr2=0x7f7f780b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.267+0000 7f7f7d521700 1 --2- 172.21.15.7:0/386642641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 0x7f7f780b5ae0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f78067900 tx=0x7f7f680091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.267+0000 7f7f7d521700 1 -- 172.21.15.7:0/386642641 shutdown_connections 2023-10-07T07:57:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.267+0000 7f7f7d521700 1 --2- 172.21.15.7:0/386642641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 0x7f7f780b5ae0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.267+0000 7f7f7d521700 1 --2- 172.21.15.7:0/386642641 >> v2:172.21.15.172:3300/0 conn(0x7f7f780d0250 0x7f7f780d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.267+0000 7f7f7d521700 1 -- 172.21.15.7:0/386642641 >> 172.21.15.7:0/386642641 conn(0x7f7f78067be0 msgr2=0x7f7f78067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.267+0000 7f7f7d521700 1 -- 172.21.15.7:0/386642641 shutdown_connections 2023-10-07T07:57:56.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.267+0000 7f7f7d521700 1 -- 172.21.15.7:0/386642641 wait complete. 2023-10-07T07:57:56.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.268+0000 7f7f7d521700 1 Processor -- start 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f7d521700 1 -- start start 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f7d521700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f780b3b60 0x7f7f7815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f7d521700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f780d0250 0x7f7f7814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f7d521700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 0x7f7f78150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f7d521700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f781598f0 con 0x7f7f78159380 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f7d521700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78159bd0 con 0x7f7f780b3b60 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f7d521700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78159eb0 con 0x7f7f780d0250 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 0x7f7f78150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 0x7f7f78150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33596/0 (socket says 172.21.15.7:33596) 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.269+0000 7f7f777fe700 1 -- 172.21.15.7:0/3326703327 learned_addr learned my addr 172.21.15.7:0/3326703327 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f76ffd700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f780b3b60 0x7f7f7815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f767fc700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f780d0250 0x7f7f7814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f777fe700 1 -- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f780d0250 msgr2=0x7f7f7814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f777fe700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f780d0250 0x7f7f7814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f777fe700 1 -- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f780b3b60 msgr2=0x7f7f7815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f777fe700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f780b3b60 0x7f7f7815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f777fe700 1 -- 172.21.15.7:0/3326703327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f68009420 con 0x7f7f78159380 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f767fc700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f780d0250 0x7f7f7814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f777fe700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 0x7f7f78150120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000cbf0 tx=0x7f7f700019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f5effd700 1 -- 172.21.15.7:0/3326703327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000a910 con 0x7f7f78159380 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f5effd700 1 -- 172.21.15.7:0/3326703327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7000aa70 con 0x7f7f78159380 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f5effd700 1 -- 172.21.15.7:0/3326703327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000ae30 con 0x7f7f78159380 2023-10-07T07:57:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7815a190 con 0x7f7f78159380 2023-10-07T07:57:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.270+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7815aed0 con 0x7f7f78159380 2023-10-07T07:57:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.271+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f780716c0 con 0x7f7f78159380 2023-10-07T07:57:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.272+0000 7f7f5effd700 1 -- 172.21.15.7:0/3326703327 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f70007300 con 0x7f7f78159380 2023-10-07T07:57:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.272+0000 7f7f5effd700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7f64066f30 0x7f7f640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.272+0000 7f7f5effd700 1 -- 172.21.15.7:0/3326703327 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7f70084870 con 0x7f7f78159380 2023-10-07T07:57:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.272+0000 7f7f76ffd700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7f64066f30 0x7f7f640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.273+0000 7f7f76ffd700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7f64066f30 0x7f7f640693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7f680091e0 tx=0x7f7f68017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.276+0000 7f7f5effd700 1 -- 172.21.15.7:0/3326703327 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f70051b60 con 0x7f7f78159380 2023-10-07T07:57:56.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.481+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f780cd120 con 0x7f7f78159380 2023-10-07T07:57:56.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.482+0000 7f7f5effd700 1 -- 172.21.15.7:0/3326703327 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7f70055180 con 0x7f7f78159380 2023-10-07T07:57:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:56.483 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:57:56.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7f64066f30 msgr2=0x7f7f640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7f64066f30 0x7f7f640693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7f680091e0 tx=0x7f7f68017000 comp rx=0 tx=0).stop 2023-10-07T07:57:56.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 msgr2=0x7f7f78150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 0x7f7f78150120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000cbf0 tx=0x7f7f700019b0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 shutdown_connections 2023-10-07T07:57:56.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7f64066f30 0x7f7f640693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f78159380 0x7f7f78150120 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f780b3b60 0x7f7f7815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 --2- 172.21.15.7:0/3326703327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f780d0250 0x7f7f7814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 >> 172.21.15.7:0/3326703327 conn(0x7f7f78067be0 msgr2=0x7f7f780b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 shutdown_connections 2023-10-07T07:57:56.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.484+0000 7f7f7d521700 1 -- 172.21.15.7:0/3326703327 wait complete. 2023-10-07T07:57:56.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-07T07:57:56.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-07T07:57:56.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-07T07:57:56.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-07T07:57:56.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-07T07:57:56.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-07T07:57:56.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-07T07:57:56.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-07T07:57:56.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.736+0000 7fdda95e7700 1 Processor -- start 2023-10-07T07:57:56.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.736+0000 7f8707697700 1 Processor -- start 2023-10-07T07:57:56.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.736+0000 7fdda95e7700 1 -- start start 2023-10-07T07:57:56.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.737+0000 7efdb0bf9700 1 Processor -- start 2023-10-07T07:57:56.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.737+0000 7efdb0bf9700 1 -- start start 2023-10-07T07:57:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.737+0000 7fdda95e7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.737+0000 7fdda95e7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdda4147330 0x7fdda40c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.737+0000 7fdda95e7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda406ae70 con 0x7fdda406a930 2023-10-07T07:57:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.737+0000 7fdda95e7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda406b150 con 0x7fdda4147330 2023-10-07T07:57:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.737+0000 7fdda95e7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda406b430 con 0x7fdda4146f60 2023-10-07T07:57:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.737+0000 7f8707697700 1 -- start start 2023-10-07T07:57:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7fdda2ffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7fdda2ffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda406a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:60732/0 (socket says 172.21.15.7:60732) 2023-10-07T07:57:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7fdda2ffd700 1 -- 172.21.15.7:0/1545056432 learned_addr learned my addr 172.21.15.7:0/1545056432 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7fdda37fe700 1 --2- 172.21.15.7:0/1545056432 >> v2:172.21.15.7:3300/0 conn(0x7fdda4147330 0x7fdda40c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7fdda2ffd700 1 -- 172.21.15.7:0/1545056432 >> v2:172.21.15.7:3300/0 conn(0x7fdda4147330 msgr2=0x7fdda40c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7fdda2ffd700 1 --2- 172.21.15.7:0/1545056432 >> v2:172.21.15.7:3300/0 conn(0x7fdda4147330 0x7fdda40c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7fdda2ffd700 1 -- 172.21.15.7:0/1545056432 >> v1:172.21.15.172:6789/0 conn(0x7fdda406a930 legacy=0x7fdda40cab80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:56.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7fdda2ffd700 1 -- 172.21.15.7:0/1545056432 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda406b710 con 0x7fdda4146f60 2023-10-07T07:57:56.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7efdb0bf9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efdac0d0250 0x7efdac0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7f8707697700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87000c92b0 0x7f870006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.738+0000 7f8707697700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8700155f20 0x7f870006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8707697700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870006b700 con 0x7f8700156390 2023-10-07T07:57:56.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8707697700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870015d120 con 0x7f87000c92b0 2023-10-07T07:57:56.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8707697700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87000c9680 con 0x7f8700155f20 2023-10-07T07:57:56.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8704c32700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8700155f20 0x7f870006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8704c32700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8700155f20 0x7f870006e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39336/0 (socket says 172.21.15.7:39336) 2023-10-07T07:57:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8704c32700 1 -- 172.21.15.7:0/1673633417 learned_addr learned my addr 172.21.15.7:0/1673633417 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7fdda2ffd700 1 --2- 172.21.15.7:0/1545056432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda406a3f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdd940099b0 tx=0x7fdd94009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8ffc4e23bb149de server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7efdb0bf9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdac0b61f0 con 0x7efdac0b3b60 2023-10-07T07:57:56.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7efdb0bf9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdac0d0be0 con 0x7efdac159380 2023-10-07T07:57:56.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7efdb0bf9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdac0b4820 con 0x7efdac0d0250 2023-10-07T07:57:56.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7fdda1ffb700 1 -- 172.21.15.7:0/1545056432 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94009ea0 con 0x7fdda4146f60 2023-10-07T07:57:56.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7fdda1ffb700 1 -- 172.21.15.7:0/1545056432 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9400b820 con 0x7fdda4146f60 2023-10-07T07:57:56.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7fdda1ffb700 1 -- 172.21.15.7:0/1545056432 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400bbc0 con 0x7fdda4146f60 2023-10-07T07:57:56.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8704c32700 1 -- 172.21.15.7:0/1673633417 >> v2:172.21.15.7:3300/0 conn(0x7f87000c92b0 msgr2=0x7f870006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8704c32700 1 --2- 172.21.15.7:0/1673633417 >> v2:172.21.15.7:3300/0 conn(0x7f87000c92b0 0x7f870006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8704c32700 1 -- 172.21.15.7:0/1673633417 >> v1:172.21.15.172:6789/0 conn(0x7f8700156390 legacy=0x7f8700159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7f8704c32700 1 -- 172.21.15.7:0/1673633417 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870006aaf0 con 0x7f8700155f20 2023-10-07T07:57:56.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7fb521ac8700 1 Processor -- start 2023-10-07T07:57:56.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7fb521ac8700 1 -- start start 2023-10-07T07:57:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7f8704c32700 1 --2- 172.21.15.7:0/1673633417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 0x7f870006e2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f870006e0b0 tx=0x7f86ec009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b9c237cc168c2a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7f86fffff700 1 -- 172.21.15.7:0/1673633417 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00b820 con 0x7f8700155f20 2023-10-07T07:57:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7f86fffff700 1 -- 172.21.15.7:0/1673633417 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec00b980 con 0x7f8700155f20 2023-10-07T07:57:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7f86fffff700 1 -- 172.21.15.7:0/1673633417 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00bd20 con 0x7f8700155f20 2023-10-07T07:57:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7f8707697700 1 -- 172.21.15.7:0/1673633417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 msgr2=0x7f870006e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7f8707697700 1 --2- 172.21.15.7:0/1673633417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 0x7f870006e2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f870006e0b0 tx=0x7f86ec009250 comp rx=0 tx=0).stop 2023-10-07T07:57:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7fb521ac8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb51c0c6be0 0x7fb51c0c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7fb521ac8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0cb040 con 0x7fb51c13b920 2023-10-07T07:57:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7fb521ac8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0ca430 con 0x7fb51c156460 2023-10-07T07:57:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7fb521ac8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c0ca710 con 0x7fb51c0c6be0 2023-10-07T07:57:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7fb51b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb51c13b920 0x7fb51c0c64d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35976/0 (socket says 172.21.15.7:35976) 2023-10-07T07:57:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7fb51b7fe700 1 -- 172.21.15.7:0/2870166758 learned_addr learned my addr 172.21.15.7:0/2870166758 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7fdda95e7700 1 -- 172.21.15.7:0/1545056432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 msgr2=0x7fdda406a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7fdda95e7700 1 --2- 172.21.15.7:0/1545056432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda406a3f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdd940099b0 tx=0x7fdd94009250 comp rx=0 tx=0).stop 2023-10-07T07:57:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7fdda95e7700 1 -- 172.21.15.7:0/1545056432 shutdown_connections 2023-10-07T07:57:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7fdda95e7700 1 --2- 172.21.15.7:0/1545056432 >> v2:172.21.15.7:3300/0 conn(0x7fdda4147330 0x7fdda40c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7fdda95e7700 1 --2- 172.21.15.7:0/1545056432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda406a3f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7fdda95e7700 1 -- 172.21.15.7:0/1545056432 >> 172.21.15.7:0/1545056432 conn(0x7fdda4068bd0 msgr2=0x7fdda40ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7efdaa59c700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7efdac159380 0x7efdac0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:59974/0 (socket says 172.21.15.7:59974) 2023-10-07T07:57:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.739+0000 7efdaa59c700 1 -- 172.21.15.7:0/1267975088 learned_addr learned my addr 172.21.15.7:0/1267975088 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.740+0000 7efda9d9b700 1 --2- 172.21.15.7:0/1267975088 >> v2:172.21.15.7:3300/0 conn(0x7efdac0d0250 0x7efdac0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7efda9d9b700 1 -- 172.21.15.7:0/1267975088 >> v1:172.21.15.172:6789/0 conn(0x7efdac159380 legacy=0x7efdac0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7efda9d9b700 1 -- 172.21.15.7:0/1267975088 >> v1:172.21.15.7:6789/0 conn(0x7efdac0b3b60 legacy=0x7efdac0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7efda9d9b700 1 -- 172.21.15.7:0/1267975088 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac0b5000 con 0x7efdac0d0250 2023-10-07T07:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7efda9d9b700 1 --2- 172.21.15.7:0/1267975088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdac0d0250 0x7efdac0d06a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efda00098e0 tx=0x7efda00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e346900ace354ee server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.741+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642136794 0 0) 0x7fb51c0cb040 con 0x7fb51c13b920 2023-10-07T07:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb504003570 con 0x7fb51c13b920 2023-10-07T07:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7efda959a700 1 -- 172.21.15.7:0/1267975088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0009e80 con 0x7efdac0d0250 2023-10-07T07:57:56.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7efda959a700 1 -- 172.21.15.7:0/1267975088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda000b7e0 con 0x7efdac0d0250 2023-10-07T07:57:56.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7efda959a700 1 -- 172.21.15.7:0/1267975088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda000bb60 con 0x7efdac0d0250 2023-10-07T07:57:56.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3224132313 0 0) 0x7fb504003570 con 0x7fb51c13b920 2023-10-07T07:57:56.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 >> v2:172.21.15.7:3300/0 conn(0x7fb51c0c6be0 msgr2=0x7fb51c0c9d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T07:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 --2- 172.21.15.7:0/2870166758 >> v2:172.21.15.7:3300/0 conn(0x7fb51c0c6be0 0x7fb51c0c9d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 >> v1:172.21.15.172:6789/0 conn(0x7fb51c156460 legacy=0x7fb51c0695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0ca9f0 con 0x7fb51c13b920 2023-10-07T07:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7fb50c001470 con 0x7fb51c13b920 2023-10-07T07:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 >> v1:172.21.15.7:6789/0 conn(0x7fb51c13b920 legacy=0x7fb51c0c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb504003a60 0x7fb504003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb504004810 0x7fb504004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 0x7fb50400cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0ca710 con 0x7fb5040053c0 2023-10-07T07:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb504005c90 con 0x7fb504004810 2023-10-07T07:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb504005e50 con 0x7fb504003a60 2023-10-07T07:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fdda95e7700 1 -- 172.21.15.7:0/1545056432 shutdown_connections 2023-10-07T07:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fdda95e7700 1 -- 172.21.15.7:0/1545056432 wait complete. 2023-10-07T07:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7fb51b7fe700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb504003a60 0x7fb504003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7efdb0bf9700 1 -- 172.21.15.7:0/1267975088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdac0d0250 msgr2=0x7efdac0d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7efdb0bf9700 1 --2- 172.21.15.7:0/1267975088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdac0d0250 0x7efdac0d06a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efda00098e0 tx=0x7efda00091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7fb51bfff700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 0x7fb50400cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7fb51bfff700 1 -- 172.21.15.7:0/2870166758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb504003a60 msgr2=0x7fb504003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7fb51bfff700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb504003a60 0x7fb504003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7fb51bfff700 1 -- 172.21.15.7:0/2870166758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb504004810 msgr2=0x7fb504004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T07:57:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7fb51bfff700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb504004810 0x7fb504004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7fb51bfff700 1 -- 172.21.15.7:0/2870166758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0ca9f0 con 0x7fb5040053c0 2023-10-07T07:57:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7fb51bfff700 1 -- 172.21.15.7:0/2870166758 reap_dead start 2023-10-07T07:57:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7efdb0bf9700 1 -- 172.21.15.7:0/1267975088 shutdown_connections 2023-10-07T07:57:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7efdb0bf9700 1 --2- 172.21.15.7:0/1267975088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdac0d0250 0x7efdac0d06a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7efdb0bf9700 1 -- 172.21.15.7:0/1267975088 >> 172.21.15.7:0/1267975088 conn(0x7efdac067be0 msgr2=0x7efdac067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7efdb0bf9700 1 -- 172.21.15.7:0/1267975088 shutdown_connections 2023-10-07T07:57:56.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.742+0000 7f8fe78bb700 1 Processor -- start 2023-10-07T07:57:56.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.744+0000 7fb51bfff700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 0x7fb50400cb70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb514001fc0 tx=0x7fb514001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45be51e98cef7822 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.744+0000 7f8707697700 1 -- 172.21.15.7:0/1673633417 shutdown_connections 2023-10-07T07:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.744+0000 7f8707697700 1 --2- 172.21.15.7:0/1673633417 >> v2:172.21.15.7:3300/0 conn(0x7f87000c92b0 0x7f870006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.744+0000 7f8707697700 1 --2- 172.21.15.7:0/1673633417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 0x7f870006e2c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.744+0000 7f8707697700 1 -- 172.21.15.7:0/1673633417 >> 172.21.15.7:0/1673633417 conn(0x7f8700068bd0 msgr2=0x7f87000ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.744+0000 7fdda95e7700 1 Processor -- start 2023-10-07T07:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.744+0000 7fdda95e7700 1 -- start start 2023-10-07T07:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda95e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda406a930 0x7fdda415d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda95e7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda95e7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda4147330 0x7fdda406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda95e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4160600 con 0x7fdda406a930 2023-10-07T07:57:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda95e7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda41608e0 con 0x7fdda4147330 2023-10-07T07:57:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7fd17c994700 1 Processor -- start 2023-10-07T07:57:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7fd17c994700 1 -- start start 2023-10-07T07:57:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7fd17c994700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd178159380 0x7fd1780b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7fd17c994700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1780d0250 0x7fd1780d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7fd17c994700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1780d0be0 con 0x7fd1780b3b60 2023-10-07T07:57:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7fd17c994700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1780b4820 con 0x7fd178159380 2023-10-07T07:57:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7fd17c994700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1780b5000 con 0x7fd1780d0250 2023-10-07T07:57:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.743+0000 7efdb0bf9700 1 -- 172.21.15.7:0/1267975088 wait complete. 2023-10-07T07:57:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7efdb0bf9700 1 Processor -- start 2023-10-07T07:57:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.744+0000 7f8fe78bb700 1 -- start start 2023-10-07T07:57:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7f8fe78bb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8fe00c6be0 0x7f8fe00cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7f8fe78bb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe015e2b0 0x7f8fe015e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7f8fe78bb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe00c9a60 con 0x7f8fe00cb3c0 2023-10-07T07:57:56.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7f8fe78bb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe0140cd0 con 0x7f8fe00c6be0 2023-10-07T07:57:56.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7f8fe78bb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00cbcd0 con 0x7f8fe015e2b0 2023-10-07T07:57:56.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda95e7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4160bc0 con 0x7fdda4146f60 2023-10-07T07:57:56.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda27fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda27fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda415fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:60758/0 (socket says 172.21.15.7:60758) 2023-10-07T07:57:56.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda27fc700 1 -- 172.21.15.7:0/3290790919 learned_addr learned my addr 172.21.15.7:0/3290790919 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda2ffd700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda406a930 0x7fdda415d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fdda37fe700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda4147330 0x7fdda406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7f8fe5e58700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe015e2b0 0x7f8fe015e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7f8fe5e58700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe015e2b0 0x7f8fe015e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:60768/0 (socket says 172.21.15.7:60768) 2023-10-07T07:57:56.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7f8fe5e58700 1 -- 172.21.15.7:0/240358417 learned_addr learned my addr 172.21.15.7:0/240358417 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fdda2ffd700 1 -- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 msgr2=0x7fdda415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fdda2ffd700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdda4146f60 0x7fdda415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fdda2ffd700 1 -- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda4147330 msgr2=0x7fdda406e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fdda2ffd700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda4147330 0x7fdda406e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fdda2ffd700 1 -- 172.21.15.7:0/3290790919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940094f0 con 0x7fdda406a930 2023-10-07T07:57:56.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7efdb0bf9700 1 -- start start 2023-10-07T07:57:56.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7efdb0bf9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdac0b3b60 0x7efdac154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7efdb0bf9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdac0d0250 0x7efdac0b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7efdb0bf9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdac159380 0x7efdac0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7efdb0bf9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac154870 con 0x7efdac159380 2023-10-07T07:57:56.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7efdb0bf9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac15b330 con 0x7efdac0d0250 2023-10-07T07:57:56.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7efdb0bf9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdac15a720 con 0x7efdac0b3b60 2023-10-07T07:57:56.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fdda2ffd700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda406a930 0x7fdda415d0d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdda406e0b0 tx=0x7fdd94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7f8fe5e58700 1 -- 172.21.15.7:0/240358417 >> v2:172.21.15.172:3300/0 conn(0x7f8fe00c6be0 msgr2=0x7f8fe00cb790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T07:57:56.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7f8fe5e58700 1 --2- 172.21.15.7:0/240358417 >> v2:172.21.15.172:3300/0 conn(0x7f8fe00c6be0 0x7f8fe00cb790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7f8fe5e58700 1 -- 172.21.15.7:0/240358417 >> v1:172.21.15.7:6789/0 conn(0x7f8fe00cb3c0 legacy=0x7f8fe00c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7f8fe5e58700 1 -- 172.21.15.7:0/240358417 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe006a240 con 0x7f8fe015e2b0 2023-10-07T07:57:56.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fdd8affd700 1 -- 172.21.15.7:0/3290790919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94011aa0 con 0x7fdda406a930 2023-10-07T07:57:56.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fdd8affd700 1 -- 172.21.15.7:0/3290790919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd94020070 con 0x7fdda406a930 2023-10-07T07:57:56.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fdd8affd700 1 -- 172.21.15.7:0/3290790919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94011e70 con 0x7fdda406a930 2023-10-07T07:57:56.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7f8fe5e58700 1 --2- 172.21.15.7:0/240358417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe015e2b0 0x7f8fe015e6d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009f20 tx=0x7f8fd8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d776dfb01cd8b05 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efdaa59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdac0b3b60 0x7efdac154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efdaa59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdac0b3b60 0x7efdac154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:60770/0 (socket says 172.21.15.7:60770) 2023-10-07T07:57:56.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efdaa59c700 1 -- 172.21.15.7:0/794319152 learned_addr learned my addr 172.21.15.7:0/794319152 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efda9d9b700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdac0d0250 0x7efdac0b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efdaad9d700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdac159380 0x7efdac0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efda9d9b700 1 -- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdac0b3b60 msgr2=0x7efdac154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efda9d9b700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdac0b3b60 0x7efdac154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efda9d9b700 1 -- 172.21.15.7:0/794319152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdac159380 msgr2=0x7efdac0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efda9d9b700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdac159380 0x7efdac0b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7efda9d9b700 1 -- 172.21.15.7:0/794319152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda0009420 con 0x7efdac0d0250 2023-10-07T07:57:56.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7efda9d9b700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdac0d0250 0x7efdac0b1610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efdac0af100 tx=0x7efda0016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7f8707697700 1 -- 172.21.15.7:0/1673633417 shutdown_connections 2023-10-07T07:57:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7fd17659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd178159380 0x7fd1780b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7fd17659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd178159380 0x7fd1780b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33650/0 (socket says 172.21.15.7:33650) 2023-10-07T07:57:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7fd17659c700 1 -- 172.21.15.7:0/1708404807 learned_addr learned my addr 172.21.15.7:0/1708404807 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.745+0000 7fd175d9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1780d0250 0x7fd1780d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fd17659c700 1 -- 172.21.15.7:0/1708404807 >> v2:172.21.15.172:3300/0 conn(0x7fd1780d0250 msgr2=0x7fd1780d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fd17659c700 1 --2- 172.21.15.7:0/1708404807 >> v2:172.21.15.172:3300/0 conn(0x7fd1780d0250 0x7fd1780d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fd17659c700 1 -- 172.21.15.7:0/1708404807 >> v1:172.21.15.172:6789/0 conn(0x7fd1780b3b60 legacy=0x7fd1780b3fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:56.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fd17659c700 1 -- 172.21.15.7:0/1708404807 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780b52e0 con 0x7fd178159380 2023-10-07T07:57:56.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.746+0000 7fd17659c700 1 --2- 172.21.15.7:0/1708404807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178159380 0x7fd1780b5ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd178067900 tx=0x7fd168009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955be202b139429e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fd17559a700 1 -- 172.21.15.7:0/1708404807 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168009e70 con 0x7fd178159380 2023-10-07T07:57:56.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.747+0000 7fd17559a700 1 -- 172.21.15.7:0/1708404807 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16800b820 con 0x7fd178159380 2023-10-07T07:57:56.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fd17559a700 1 -- 172.21.15.7:0/1708404807 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16800bbc0 con 0x7fd178159380 2023-10-07T07:57:56.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514001950 con 0x7fb5040053c0 2023-10-07T07:57:56.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda4160ea0 con 0x7fdda406a930 2023-10-07T07:57:56.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.749+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda415d610 con 0x7fdda406a930 2023-10-07T07:57:56.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7f8fd7fff700 1 -- 172.21.15.7:0/240358417 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800e070 con 0x7f8fe015e2b0 2023-10-07T07:57:56.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7f8fd7fff700 1 -- 172.21.15.7:0/240358417 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd800ba40 con 0x7f8fe015e2b0 2023-10-07T07:57:56.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7f8fd7fff700 1 -- 172.21.15.7:0/240358417 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80092e0 con 0x7f8fe015e2b0 2023-10-07T07:57:56.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.749+0000 7f8fe78bb700 1 -- 172.21.15.7:0/240358417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe015e2b0 msgr2=0x7f8fe015e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.749+0000 7f8fe78bb700 1 --2- 172.21.15.7:0/240358417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe015e2b0 0x7f8fe015e6d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009f20 tx=0x7f8fd8009840 comp rx=0 tx=0).stop 2023-10-07T07:57:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51400c8b0 con 0x7fb5040053c0 2023-10-07T07:57:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fb51a7fc700 1 -- 172.21.15.7:0/2870166758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400cc50 con 0x7fb5040053c0 2023-10-07T07:57:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.749+0000 7fb521ac8700 1 -- 172.21.15.7:0/2870166758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 msgr2=0x7fb50400cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.749+0000 7fb521ac8700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 0x7fb50400cb70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb514001fc0 tx=0x7fb514001640 comp rx=0 tx=0).stop 2023-10-07T07:57:56.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fd17c994700 1 -- 172.21.15.7:0/1708404807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178159380 msgr2=0x7fd1780b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fd17c994700 1 --2- 172.21.15.7:0/1708404807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178159380 0x7fd1780b5ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd178067900 tx=0x7fd168009250 comp rx=0 tx=0).stop 2023-10-07T07:57:56.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fd17c994700 1 -- 172.21.15.7:0/1708404807 shutdown_connections 2023-10-07T07:57:56.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fd17c994700 1 --2- 172.21.15.7:0/1708404807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178159380 0x7fd1780b5ae0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd178067900 tx=0x7fd168009250 comp rx=0 tx=0).stop 2023-10-07T07:57:56.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fd17c994700 1 --2- 172.21.15.7:0/1708404807 >> v2:172.21.15.172:3300/0 conn(0x7fd1780d0250 0x7fd1780d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7fd17c994700 1 -- 172.21.15.7:0/1708404807 >> 172.21.15.7:0/1708404807 conn(0x7fd178067be0 msgr2=0x7fd178067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.749+0000 7fd17c994700 1 -- 172.21.15.7:0/1708404807 shutdown_connections 2023-10-07T07:57:56.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.749+0000 7fd17c994700 1 -- 172.21.15.7:0/1708404807 wait complete. 2023-10-07T07:57:56.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.748+0000 7f8707697700 1 -- 172.21.15.7:0/1673633417 wait complete. 2023-10-07T07:57:56.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7f8707697700 1 Processor -- start 2023-10-07T07:57:56.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7fb521ac8700 1 -- 172.21.15.7:0/2870166758 shutdown_connections 2023-10-07T07:57:56.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7fb521ac8700 1 --2- 172.21.15.7:0/2870166758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 0x7fb50400cb70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7fb521ac8700 1 -- 172.21.15.7:0/2870166758 >> 172.21.15.7:0/2870166758 conn(0x7fb51c067a80 msgr2=0x7fb51c0cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7efd9a7fc700 1 -- 172.21.15.7:0/794319152 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda000b950 con 0x7efdac0d0250 2023-10-07T07:57:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7efd9a7fc700 1 -- 172.21.15.7:0/794319152 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda0020070 con 0x7efdac0d0250 2023-10-07T07:57:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7efd9a7fc700 1 -- 172.21.15.7:0/794319152 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda001b450 con 0x7efdac0d0250 2023-10-07T07:57:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac15aa00 con 0x7efdac0d0250 2023-10-07T07:57:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac15afc0 con 0x7efdac0d0250 2023-10-07T07:57:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7f8fe78bb700 1 -- 172.21.15.7:0/240358417 shutdown_connections 2023-10-07T07:57:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7f8fe78bb700 1 --2- 172.21.15.7:0/240358417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe015e2b0 0x7f8fe015e6d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7f8fe78bb700 1 --2- 172.21.15.7:0/240358417 >> v2:172.21.15.172:3300/0 conn(0x7f8fe00c6be0 0x7f8fe00cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7f8fe78bb700 1 -- 172.21.15.7:0/240358417 >> 172.21.15.7:0/240358417 conn(0x7f8fe0067a80 msgr2=0x7f8fe00cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7f8707697700 1 -- start start 2023-10-07T07:57:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8707697700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87000c92b0 0x7f870013a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8707697700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 0x7f87000afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8707697700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700156390 0x7f87000b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8707697700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000b17d0 con 0x7f8700156390 2023-10-07T07:57:56.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8707697700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000b0730 con 0x7f8700155f20 2023-10-07T07:57:56.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8707697700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000b0a10 con 0x7f87000c92b0 2023-10-07T07:57:56.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8704c32700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 0x7f87000afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8704c32700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 0x7f87000afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39386/0 (socket says 172.21.15.7:39386) 2023-10-07T07:57:56.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8704c32700 1 -- 172.21.15.7:0/1573259791 learned_addr learned my addr 172.21.15.7:0/1573259791 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8705433700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87000c92b0 0x7f870013a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7fb521ac8700 1 -- 172.21.15.7:0/2870166758 shutdown_connections 2023-10-07T07:57:56.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.750+0000 7fd17c994700 1 Processor -- start 2023-10-07T07:57:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.751+0000 7fd17c994700 1 -- start start 2023-10-07T07:57:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.751+0000 7fd17c994700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1780b3b60 0x7fd17813dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.751+0000 7fd17c994700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1780d0250 0x7fd17814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.751+0000 7fd17c994700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd17814e420 0x7fd178154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.751+0000 7fd17c994700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178151350 con 0x7fd17814e420 2023-10-07T07:57:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.751+0000 7fd17c994700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1781505c0 con 0x7fd1780d0250 2023-10-07T07:57:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.751+0000 7fd17c994700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17814d6f0 con 0x7fd1780b3b60 2023-10-07T07:57:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7fd175d9b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1780d0250 0x7fd17814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7fd17659c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1780b3b60 0x7fd17813dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd90000fd0 con 0x7efdac0d0250 2023-10-07T07:57:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8fe78bb700 1 -- 172.21.15.7:0/240358417 shutdown_connections 2023-10-07T07:57:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8704c32700 1 -- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87000c92b0 msgr2=0x7f870013a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8704c32700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87000c92b0 0x7f870013a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8704c32700 1 -- 172.21.15.7:0/1573259791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700156390 msgr2=0x7f87000b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8704c32700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700156390 0x7f87000b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7fd175d9b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1780d0250 0x7fd17814dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39374/0 (socket says 172.21.15.7:39374) 2023-10-07T07:57:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7fd175d9b700 1 -- 172.21.15.7:0/2788934275 learned_addr learned my addr 172.21.15.7:0/2788934275 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8704c32700 1 -- 172.21.15.7:0/1573259791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0094f0 con 0x7f8700155f20 2023-10-07T07:57:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.753+0000 7f8704c32700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 0x7f87000afd50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0090b0 tx=0x7f86ec016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.753+0000 7f86fd7fa700 1 -- 172.21.15.7:0/1573259791 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec011de0 con 0x7f8700155f20 2023-10-07T07:57:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.753+0000 7f86fd7fa700 1 -- 172.21.15.7:0/1573259791 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec01f070 con 0x7f8700155f20 2023-10-07T07:57:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.753+0000 7f86fd7fa700 1 -- 172.21.15.7:0/1573259791 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0266d0 con 0x7f8700155f20 2023-10-07T07:57:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7f8fe78bb700 1 -- 172.21.15.7:0/240358417 wait complete. 2023-10-07T07:57:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.754+0000 7f8fe78bb700 1 Processor -- start 2023-10-07T07:57:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.754+0000 7efd9a7fc700 1 -- 172.21.15.7:0/794319152 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efda0022030 con 0x7efdac0d0250 2023-10-07T07:57:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7efd9a7fc700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efd8c067080 0x7efd8c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7efd9a7fc700 1 -- 172.21.15.7:0/794319152 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efda0086950 con 0x7efdac0d0250 2023-10-07T07:57:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7efd9a7fc700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7efd8c06f290 0x7efd8c0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7efd9a7fc700 1 -- 172.21.15.7:0/794319152 --> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd8c071d50 con 0x7efd8c06f290 2023-10-07T07:57:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7efd9a7fc700 1 -- 172.21.15.7:0/794319152 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7efda0086c80 con 0x7efdac0d0250 2023-10-07T07:57:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7efdaa59c700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efd8c067080 0x7efd8c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7efdaad9d700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7efd8c06f290 0x7efd8c0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.754+0000 7f8fe78bb700 1 -- start start 2023-10-07T07:57:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe78bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe00c6be0 0x7f8fe00ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.752+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdda40716c0 con 0x7fdda406a930 2023-10-07T07:57:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.754+0000 7fdd8affd700 1 -- 172.21.15.7:0/3290790919 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd94022030 con 0x7fdda406a930 2023-10-07T07:57:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.754+0000 7fdd8affd700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdd90067000 0x7fdd900694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7fdd8affd700 1 -- 172.21.15.7:0/3290790919 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd940867d0 con 0x7fdda406a930 2023-10-07T07:57:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7fdd8affd700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7fdd9006f210 0x7fdd90071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7fdd8affd700 1 -- 172.21.15.7:0/3290790919 --> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd90071cd0 con 0x7fdd9006f210 2023-10-07T07:57:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7fdd8affd700 1 -- 172.21.15.7:0/3290790919 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd94086ac0 con 0x7fdda406a930 2023-10-07T07:57:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7fdda37fe700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7fdd9006f210 0x7fdd90071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7efdaad9d700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7efd8c06f290 0x7efd8c0716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7efdaa59c700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efd8c067080 0x7efd8c069530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007880 tx=0x7efd9c004a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe78bb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe00cb3c0 0x7f8fe0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe78bb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fe015e2b0 0x7f8fe015f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe78bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe0156bc0 con 0x7f8fe00c6be0 2023-10-07T07:57:56.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe78bb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe015bd20 con 0x7f8fe015e2b0 2023-10-07T07:57:56.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe78bb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe015c000 con 0x7f8fe00cb3c0 2023-10-07T07:57:56.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe5657700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe00c6be0 0x7f8fe00ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe5657700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe00c6be0 0x7f8fe00ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33698/0 (socket says 172.21.15.7:33698) 2023-10-07T07:57:56.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe5657700 1 -- 172.21.15.7:0/762594750 learned_addr learned my addr 172.21.15.7:0/762594750 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe4e56700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe00cb3c0 0x7f8fe0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7fb521ac8700 1 -- 172.21.15.7:0/2870166758 wait complete. 2023-10-07T07:57:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.756+0000 7f8fe5657700 1 -- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe00cb3c0 msgr2=0x7f8fe0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7f8fe5657700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fe00cb3c0 0x7f8fe0156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7f8fe5657700 1 -- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fe015e2b0 msgr2=0x7f8fe015f680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-07T07:57:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7f8fe5657700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fe015e2b0 0x7f8fe015f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7f8fe5657700 1 -- 172.21.15.7:0/762594750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8009ae0 con 0x7f8fe00c6be0 2023-10-07T07:57:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7fd176d9d700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd17814e420 0x7fd178154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7f8fe5657700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe00c6be0 0x7f8fe00ae0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000b600 tx=0x7f8fd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7fb521ac8700 1 Processor -- start 2023-10-07T07:57:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7fb521ac8700 1 -- start start 2023-10-07T07:57:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7fdda27fc700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdd90067000 0x7fdd900694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7fd176d9d700 1 -- 172.21.15.7:0/2788934275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1780b3b60 msgr2=0x7fd17813dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7fd176d9d700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1780b3b60 0x7fd17813dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7fd176d9d700 1 -- 172.21.15.7:0/2788934275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1780d0250 msgr2=0x7fd17814dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7fd176d9d700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1780d0250 0x7fd17814dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7fd176d9d700 1 -- 172.21.15.7:0/2788934275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1680094f0 con 0x7fd17814e420 2023-10-07T07:57:56.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7efd9a7fc700 1 -- 172.21.15.7:0/794319152 <== osd.4 v2:172.21.15.172:6800/120243 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7efd8c071d50 con 0x7efd8c06f290 2023-10-07T07:57:56.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fd176d9d700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd17814e420 0x7fd178154cb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b4c0 tx=0x7fd16c009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.754+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87000b0cf0 con 0x7f8700155f20 2023-10-07T07:57:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.754+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87000b12b0 con 0x7f8700155f20 2023-10-07T07:57:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fdda37fe700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7fdd9006f210 0x7fdd90071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.755+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87000716c0 con 0x7f8700155f20 2023-10-07T07:57:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7f86fd7fa700 1 -- 172.21.15.7:0/1573259791 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86ec01a440 con 0x7f8700155f20 2023-10-07T07:57:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.757+0000 7f86fd7fa700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f86e4066f00 0x7f86e40693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7f86fd7fa700 1 -- 172.21.15.7:0/1573259791 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f86ec021070 con 0x7f8700155f20 2023-10-07T07:57:56.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7f8705433700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f86e4066f00 0x7f86e40693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fd1667fc700 1 -- 172.21.15.7:0/2788934275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00f510 con 0x7fd17814e420 2023-10-07T07:57:56.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fd1667fc700 1 -- 172.21.15.7:0/2788934275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c00f690 con 0x7fd17814e420 2023-10-07T07:57:56.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fd1667fc700 1 -- 172.21.15.7:0/2788934275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00fa50 con 0x7fd17814e420 2023-10-07T07:57:56.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1781507d0 con 0x7fd17814e420 2023-10-07T07:57:56.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178150d90 con 0x7fd17814e420 2023-10-07T07:57:56.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7f86fd7fa700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f86e406f090 0x7f86e40714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7f86fd7fa700 1 -- 172.21.15.7:0/1573259791 --> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86e4071b50 con 0x7f86e406f090 2023-10-07T07:57:56.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7f86fd7fa700 1 -- 172.21.15.7:0/1573259791 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86ec053980 con 0x7f8700155f20 2023-10-07T07:57:56.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7f8705c34700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f86e406f090 0x7f86e40714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fb521ac8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 0x7fb51c0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fb521ac8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb51c0c57d0 0x7fb51c154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fb521ac8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb51c179bb0 0x7fb51c0c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fb521ac8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0c4730 con 0x7fb5040053c0 2023-10-07T07:57:56.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fb521ac8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0c4a10 con 0x7fb51c0c57d0 2023-10-07T07:57:56.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fb521ac8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0c4cf0 con 0x7fb51c179bb0 2023-10-07T07:57:56.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7fdda27fc700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdd90067000 0x7fdd900694b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdda40b2e00 tx=0x7fdd9c009810 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.759+0000 7fdd8affd700 1 -- 172.21.15.7:0/3290790919 <== osd.2 v2:172.21.15.7:6824/120011 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdd90071cd0 con 0x7fdd9006f210 2023-10-07T07:57:56.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.759+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1780716c0 con 0x7fd17814e420 2023-10-07T07:57:56.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7f8fd57fa700 1 -- 172.21.15.7:0/762594750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd000f510 con 0x7f8fe00c6be0 2023-10-07T07:57:56.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7f8fd57fa700 1 -- 172.21.15.7:0/762594750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd000f690 con 0x7f8fe00c6be0 2023-10-07T07:57:56.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.758+0000 7f8fd57fa700 1 -- 172.21.15.7:0/762594750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd000fa50 con 0x7f8fe00c6be0 2023-10-07T07:57:56.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.759+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe015c2e0 con 0x7f8fe00c6be0 2023-10-07T07:57:56.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.759+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe015c8a0 con 0x7f8fe00c6be0 2023-10-07T07:57:56.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.759+0000 7f8705433700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f86e4066f00 0x7f86e40693b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007580 tx=0x7f86f4009780 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.760+0000 7f8705c34700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f86e406f090 0x7f86e40714a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.760+0000 7fb51affd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb51c0c57d0 0x7fb51c154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.760+0000 7fb51affd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb51c0c57d0 0x7fb51c154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39406/0 (socket says 172.21.15.7:39406) 2023-10-07T07:57:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.760+0000 7fb51affd700 1 -- 172.21.15.7:0/1284161143 learned_addr learned my addr 172.21.15.7:0/1284161143 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.760+0000 7fb51bfff700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb51c179bb0 0x7fb51c0c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.760+0000 7fb51b7fe700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 0x7fb51c0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.760+0000 7fb51affd700 1 -- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb51c179bb0 msgr2=0x7fb51c0c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.760+0000 7fb51affd700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb51c179bb0 0x7fb51c0c2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.761+0000 7fb51affd700 1 -- 172.21.15.7:0/1284161143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 msgr2=0x7fb51c0c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.761+0000 7fb51affd700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 0x7fb51c0c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.761+0000 7fb51affd700 1 -- 172.21.15.7:0/1284161143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0c4fd0 con 0x7fb51c0c57d0 2023-10-07T07:57:56.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.761+0000 7fb51b7fe700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5040053c0 0x7fb51c0c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:57:56.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.761+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8fbc000fd0 con 0x7f8fe00c6be0 2023-10-07T07:57:56.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.761+0000 7fd1667fc700 1 -- 172.21.15.7:0/2788934275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd16c024430 con 0x7fd17814e420 2023-10-07T07:57:56.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.761+0000 7f86fd7fa700 1 -- 172.21.15.7:0/1573259791 <== osd.0 v2:172.21.15.7:6809/120000 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f86e4071b50 con 0x7f86e406f090 2023-10-07T07:57:56.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.761+0000 7fb51affd700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb51c0c57d0 0x7fb51c154c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb51000b460 tx=0x7fb510006f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7f8fd57fa700 1 -- 172.21.15.7:0/762594750 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8fd001a400 con 0x7f8fe00c6be0 2023-10-07T07:57:56.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7f8fd57fa700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8fc4067050 0x7f8fc4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7f8fd57fa700 1 -- 172.21.15.7:0/762594750 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8fd0084050 con 0x7f8fe00c6be0 2023-10-07T07:57:56.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7f8fd57fa700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f8fc406f260 0x7f8fc4071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7f8fd57fa700 1 -- 172.21.15.7:0/762594750 --> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fc4071d20 con 0x7f8fc406f260 2023-10-07T07:57:56.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7f8fd57fa700 1 -- 172.21.15.7:0/762594750 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd00865f0 con 0x7f8fe00c6be0 2023-10-07T07:57:56.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7fb5037fe700 1 -- 172.21.15.7:0/1284161143 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000f510 con 0x7fb51c0c57d0 2023-10-07T07:57:56.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7fb5037fe700 1 -- 172.21.15.7:0/1284161143 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51000f670 con 0x7fb51c0c57d0 2023-10-07T07:57:56.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7fb5037fe700 1 -- 172.21.15.7:0/1284161143 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000fa30 con 0x7fb51c0c57d0 2023-10-07T07:57:56.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.762+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c0c52b0 con 0x7fb51c0c57d0 2023-10-07T07:57:56.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.763+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c1699b0 con 0x7fb51c0c57d0 2023-10-07T07:57:56.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.763+0000 7f8fe4e56700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8fc4067050 0x7f8fc4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.763+0000 7fd1667fc700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd158067080 0x7fd158069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.763+0000 7fd1667fc700 1 -- 172.21.15.7:0/2788934275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd16c084a30 con 0x7fd17814e420 2023-10-07T07:57:56.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.763+0000 7fd17659c700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd158067080 0x7fd158069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.763+0000 7fd1667fc700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7fd15806f290 0x7fd1580716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.763+0000 7fd1667fc700 1 -- 172.21.15.7:0/2788934275 --> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd158071d50 con 0x7fd15806f290 2023-10-07T07:57:56.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.764+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb51c0c4b80 con 0x7fb51c0c57d0 2023-10-07T07:57:56.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.764+0000 7f8fe5e58700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f8fc406f260 0x7f8fc4071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.764+0000 7fd175d9b700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7fd15806f290 0x7fd1580716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.764+0000 7fd175d9b700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7fd15806f290 0x7fd1580716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.765+0000 7fd1667fc700 1 -- 172.21.15.7:0/2788934275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd16c084d60 con 0x7fd17814e420 2023-10-07T07:57:56.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.765+0000 7f8fe5e58700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f8fc406f260 0x7f8fc4071670 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.765+0000 7f8fe4e56700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8fc4067050 0x7f8fc4069500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006fd0 tx=0x7f8fdc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.766+0000 7fd1667fc700 1 -- 172.21.15.7:0/2788934275 <== osd.3 v2:172.21.15.7:6800/120001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd158071d50 con 0x7fd15806f290 2023-10-07T07:57:56.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.766+0000 7fd17659c700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd158067080 0x7fd158069530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd178067900 tx=0x7fd168016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.766+0000 7fb5037fe700 1 -- 172.21.15.7:0/1284161143 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb51001a400 con 0x7fb51c0c57d0 2023-10-07T07:57:56.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.766+0000 7fb5037fe700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb508067080 0x7fb508069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.766+0000 7fb5037fe700 1 -- 172.21.15.7:0/1284161143 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb510083810 con 0x7fb51c0c57d0 2023-10-07T07:57:56.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.766+0000 7fb5037fe700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7fb50806f210 0x7fb508071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.766+0000 7fb5037fe700 1 -- 172.21.15.7:0/1284161143 --> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb508071cd0 con 0x7fb50806f210 2023-10-07T07:57:56.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.767+0000 7f0212795700 1 Processor -- start 2023-10-07T07:57:56.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.767+0000 7f0212795700 1 -- start start 2023-10-07T07:57:56.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.768+0000 7f0212795700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0c9fb0 0x7f020c0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.768+0000 7f0212795700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f020c0c8cd0 0x7f020c0670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.768+0000 7f0212795700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c0af590 con 0x7f020c156050 2023-10-07T07:57:56.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.768+0000 7f0212795700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0b03f0 con 0x7f020c0c8cd0 2023-10-07T07:57:56.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.768+0000 7f0212795700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0af7e0 con 0x7f020c0c9fb0 2023-10-07T07:57:56.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.768+0000 7fb51b7fe700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb508067080 0x7fb508069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.769+0000 7fb51bfff700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7fb50806f210 0x7fb508071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.769+0000 7fb5037fe700 1 -- 172.21.15.7:0/1284161143 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb510009110 con 0x7fb51c0c57d0 2023-10-07T07:57:56.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.769+0000 7f020b7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0c9fb0 0x7f020c0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.769+0000 7f020b7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0c9fb0 0x7f020c0aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:60818/0 (socket says 172.21.15.7:60818) 2023-10-07T07:57:56.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.769+0000 7f020b7fe700 1 -- 172.21.15.7:0/3722178713 learned_addr learned my addr 172.21.15.7:0/3722178713 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.769+0000 7fb51bfff700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7fb50806f210 0x7fb508071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.769+0000 7f0210d32700 1 --2- 172.21.15.7:0/3722178713 >> v2:172.21.15.172:3300/0 conn(0x7f020c0c8cd0 0x7f020c0670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7fb5037fe700 1 -- 172.21.15.7:0/1284161143 <== osd.1 v2:172.21.15.7:6808/120002 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb508071cd0 con 0x7fb50806f210 2023-10-07T07:57:56.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7f8fd57fa700 1 -- 172.21.15.7:0/762594750 <== osd.5 v2:172.21.15.172:6808/120244 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8fc4071d20 con 0x7f8fc406f260 2023-10-07T07:57:56.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7f0210d32700 1 -- 172.21.15.7:0/3722178713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0c9fb0 msgr2=0x7f020c0aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7f0210d32700 1 --2- 172.21.15.7:0/3722178713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0c9fb0 0x7f020c0aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7f0210d32700 1 -- 172.21.15.7:0/3722178713 >> v1:172.21.15.172:6789/0 conn(0x7f020c156050 legacy=0x7f020c0c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7f0210d32700 1 -- 172.21.15.7:0/3722178713 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0afac0 con 0x7f020c0c8cd0 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7f0210d32700 1 --2- 172.21.15.7:0/3722178713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 0x7f020c0670f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01f40098e0 tx=0x7f01f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd99d39fc5f94d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7f020affd700 1 -- 172.21.15.7:0/3722178713 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4011630 con 0x7f020c0c8cd0 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7f020affd700 1 -- 172.21.15.7:0/3722178713 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f4011790 con 0x7f020c0c8cd0 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.771+0000 7f020affd700 1 -- 172.21.15.7:0/3722178713 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4011b30 con 0x7f020c0c8cd0 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.776+0000 7f0212795700 1 -- 172.21.15.7:0/3722178713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 msgr2=0x7f020c0670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.776+0000 7f0212795700 1 --2- 172.21.15.7:0/3722178713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 0x7f020c0670f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01f40098e0 tx=0x7f01f40091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.776+0000 7f0212795700 1 -- 172.21.15.7:0/3722178713 shutdown_connections 2023-10-07T07:57:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.776+0000 7f0212795700 1 --2- 172.21.15.7:0/3722178713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 0x7f020c0670f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.776+0000 7f0212795700 1 --2- 172.21.15.7:0/3722178713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c0c9fb0 0x7f020c0aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.776+0000 7f0212795700 1 -- 172.21.15.7:0/3722178713 >> 172.21.15.7:0/3722178713 conn(0x7f020c0cd1a0 msgr2=0x7f020c0c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.776+0000 7f0212795700 1 -- 172.21.15.7:0/3722178713 shutdown_connections 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.776+0000 7f0212795700 1 -- 172.21.15.7:0/3722178713 wait complete. 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.778+0000 7f0212795700 1 Processor -- start 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.780+0000 7f0212795700 1 -- start start 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.781+0000 7f0212795700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 0x7f020c14b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.781+0000 7f0212795700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f020c0c9fb0 0x7f020c0c1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.781+0000 7f0212795700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c156050 0x7f020c0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.781+0000 7f0212795700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c14b7e0 con 0x7f020c0c9fb0 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.781+0000 7f0212795700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c14ca20 con 0x7f020c0c8cd0 2023-10-07T07:57:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.781+0000 7f0212795700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0c4070 con 0x7f020c156050 2023-10-07T07:57:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7fb51b7fe700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb508067080 0x7fb508069530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb50c001bb0 tx=0x7fb50c006040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.781+0000 7f020bfff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 0x7f020c14b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.781+0000 7f020bfff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 0x7f020c14b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39418/0 (socket says 172.21.15.7:39418) 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.781+0000 7f020bfff700 1 -- 172.21.15.7:0/1463376586 learned_addr learned my addr 172.21.15.7:0/1463376586 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7f020bfff700 1 -- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c156050 msgr2=0x7f020c0cb000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7f020bfff700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f020c156050 0x7f020c0cb000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7f020bfff700 1 -- 172.21.15.7:0/1463376586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f020c0c9fb0 msgr2=0x7f020c0c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7f020bfff700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f020c0c9fb0 0x7f020c0c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7f020bfff700 1 -- 172.21.15.7:0/1463376586 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4009420 con 0x7f020c0c8cd0 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7f020bfff700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 0x7f020c14b2a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b610 tx=0x7f01fc009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7f0203fff700 1 -- 172.21.15.7:0/1463376586 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00f510 con 0x7f020c0c8cd0 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7f0203fff700 1 -- 172.21.15.7:0/1463376586 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc00f690 con 0x7f020c0c8cd0 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.782+0000 7f0203fff700 1 -- 172.21.15.7:0/1463376586 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00fa50 con 0x7f020c0c8cd0 2023-10-07T07:57:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.783+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c0c1b70 con 0x7f020c0c8cd0 2023-10-07T07:57:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.783+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c14be10 con 0x7f020c0c8cd0 2023-10-07T07:57:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.785+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 --> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd90002d30 con 0x7efd8c06f290 2023-10-07T07:57:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.784+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f020c0cd6c0 con 0x7f020c0c8cd0 2023-10-07T07:57:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.785+0000 7f0203fff700 1 -- 172.21.15.7:0/1463376586 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01fc01a400 con 0x7f020c0c8cd0 2023-10-07T07:57:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.786+0000 7efd9a7fc700 1 -- 172.21.15.7:0/794319152 <== osd.4 v2:172.21.15.172:6800/120243 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efd90002d30 con 0x7efd8c06f290 2023-10-07T07:57:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.786+0000 7f0203fff700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f01ec066f30 0x7f01ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.786+0000 7f0203fff700 1 -- 172.21.15.7:0/1463376586 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f01fc0840c0 con 0x7f020c0c8cd0 2023-10-07T07:57:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.786+0000 7f0203fff700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f01ec06f140 0x7f01ec071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.786+0000 7f0203fff700 1 -- 172.21.15.7:0/1463376586 --> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f01ec071c00 con 0x7f01ec06f140 2023-10-07T07:57:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.786+0000 7f0203fff700 1 -- 172.21.15.7:0/1463376586 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01fc086620 con 0x7f020c0c8cd0 2023-10-07T07:57:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.786+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7efd8c06f290 msgr2=0x7efd8c0716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.786+0000 7efdb0bf9700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7efd8c06f290 0x7efd8c0716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.786+0000 7f0210d32700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f01ec06f140 0x7f01ec071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.787+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efd8c067080 msgr2=0x7efd8c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.787+0000 7efdb0bf9700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efd8c067080 0x7efd8c069530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007880 tx=0x7efd9c004a80 comp rx=0 tx=0).stop 2023-10-07T07:57:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.787+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdac0d0250 msgr2=0x7efdac0b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.787+0000 7efdb0bf9700 1 --2- 172.21.15.7:0/794319152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdac0d0250 0x7efdac0b1610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efdac0af100 tx=0x7efda0016000 comp rx=0 tx=0).stop 2023-10-07T07:57:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.787+0000 7efdaad9d700 1 -- 172.21.15.7:0/794319152 reap_dead start 2023-10-07T07:57:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.787+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 shutdown_connections 2023-10-07T07:57:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.787+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 >> 172.21.15.7:0/794319152 conn(0x7efdac067be0 msgr2=0x7efdac157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.787+0000 7f0210d32700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f01ec06f140 0x7f01ec071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.788+0000 7f0203fff700 1 -- 172.21.15.7:0/1463376586 <== osd.7 v2:172.21.15.172:6824/120274 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f01ec071c00 con 0x7f01ec06f140 2023-10-07T07:57:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.788+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 --> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb51c0716c0 con 0x7fb50806f210 2023-10-07T07:57:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.790+0000 7fb5037fe700 1 -- 172.21.15.7:0/1284161143 <== osd.1 v2:172.21.15.7:6808/120002 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb51c0716c0 con 0x7fb50806f210 2023-10-07T07:57:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.790+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7fb50806f210 msgr2=0x7fb508071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.790+0000 7fb521ac8700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7fb50806f210 0x7fb508071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.790+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb508067080 msgr2=0x7fb508069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.790+0000 7fb521ac8700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb508067080 0x7fb508069530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb50c001bb0 tx=0x7fb50c006040 comp rx=0 tx=0).stop 2023-10-07T07:57:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.790+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb51c0c57d0 msgr2=0x7fb51c154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.790+0000 7fb521ac8700 1 --2- 172.21.15.7:0/1284161143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb51c0c57d0 0x7fb51c154c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb51000b460 tx=0x7fb510006f70 comp rx=0 tx=0).stop 2023-10-07T07:57:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.791+0000 7fb51bfff700 1 -- 172.21.15.7:0/1284161143 reap_dead start 2023-10-07T07:57:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.791+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 shutdown_connections 2023-10-07T07:57:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.791+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 >> 172.21.15.7:0/1284161143 conn(0x7fb51c067a80 msgr2=0x7fb51c1576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.791+0000 7f020b7fe700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f01ec066f30 0x7f01ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.794+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 shutdown_connections 2023-10-07T07:57:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.791+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 shutdown_connections 2023-10-07T07:57:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.794+0000 7fb521ac8700 1 -- 172.21.15.7:0/1284161143 wait complete. 2023-10-07T07:57:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.794+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 --> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdda4160a50 con 0x7fdd9006f210 2023-10-07T07:57:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.794+0000 7f35b2821700 1 Processor -- start 2023-10-07T07:57:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.794+0000 7f35b2821700 1 -- start start 2023-10-07T07:57:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.794+0000 7efdb0bf9700 1 -- 172.21.15.7:0/794319152 wait complete. 2023-10-07T07:57:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.795+0000 7f35b2821700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f35ac06e360 0x7f35ac0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35b2821700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35ac0cab80 0x7f35ac0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35b2821700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac0c6700 con 0x7f35ac15b3e0 2023-10-07T07:57:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35b2821700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac0cbca0 con 0x7f35ac06e360 2023-10-07T07:57:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35b2821700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac15b7b0 con 0x7f35ac0cab80 2023-10-07T07:57:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35abfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f35ac15b3e0 0x7f35ac06dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35996/0 (socket says 172.21.15.7:35996) 2023-10-07T07:57:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35abfff700 1 -- 172.21.15.7:0/4181296119 learned_addr learned my addr 172.21.15.7:0/4181296119 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35ab7fe700 1 --2- 172.21.15.7:0/4181296119 >> v2:172.21.15.172:3300/0 conn(0x7f35ac06e360 0x7f35ac0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35ab7fe700 1 -- 172.21.15.7:0/4181296119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35ac0cab80 msgr2=0x7f35ac0c6b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35ab7fe700 1 --2- 172.21.15.7:0/4181296119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35ac0cab80 0x7f35ac0c6b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35ab7fe700 1 -- 172.21.15.7:0/4181296119 >> v1:172.21.15.7:6789/0 conn(0x7f35ac15b3e0 legacy=0x7f35ac06dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.796+0000 7f35ab7fe700 1 -- 172.21.15.7:0/4181296119 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac0cb400 con 0x7f35ac06e360 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.797+0000 7f35ab7fe700 1 --2- 172.21.15.7:0/4181296119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac06e360 0x7f35ac0ca640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35a00098e0 tx=0x7f35a00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6928cac5c58f61b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.797+0000 7f35aaffd700 1 -- 172.21.15.7:0/4181296119 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0009e80 con 0x7f35ac06e360 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.797+0000 7f35aaffd700 1 -- 172.21.15.7:0/4181296119 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a000b860 con 0x7f35ac06e360 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.797+0000 7f35aaffd700 1 -- 172.21.15.7:0/4181296119 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000bc20 con 0x7f35ac06e360 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f35b2821700 1 -- 172.21.15.7:0/4181296119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac06e360 msgr2=0x7f35ac0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f35b2821700 1 --2- 172.21.15.7:0/4181296119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac06e360 0x7f35ac0ca640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35a00098e0 tx=0x7f35a00091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f35b2821700 1 -- 172.21.15.7:0/4181296119 shutdown_connections 2023-10-07T07:57:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f35b2821700 1 --2- 172.21.15.7:0/4181296119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35ac0cab80 0x7f35ac0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f35b2821700 1 --2- 172.21.15.7:0/4181296119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac06e360 0x7f35ac0ca640 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f35b2821700 1 -- 172.21.15.7:0/4181296119 >> 172.21.15.7:0/4181296119 conn(0x7f35ac0cbe90 msgr2=0x7f35ac0cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f35b2821700 1 -- 172.21.15.7:0/4181296119 shutdown_connections 2023-10-07T07:57:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 --> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f870006c340 con 0x7f86e406f090 2023-10-07T07:57:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.799+0000 7fdd8affd700 1 -- 172.21.15.7:0/3290790919 <== osd.2 v2:172.21.15.7:6824/120011 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdda4160a50 con 0x7fdd9006f210 2023-10-07T07:57:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.799+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 --> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd17806f3f0 con 0x7fd15806f290 2023-10-07T07:57:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.799+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7fdd9006f210 msgr2=0x7fdd90071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.799+0000 7fdda95e7700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7fdd9006f210 0x7fdd90071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.800+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdd90067000 msgr2=0x7fdd900694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.800+0000 7fdda95e7700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdd90067000 0x7fdd900694b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdda40b2e00 tx=0x7fdd9c009810 comp rx=0 tx=0).stop 2023-10-07T07:57:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.800+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda406a930 msgr2=0x7fdda415d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.800+0000 7fdda95e7700 1 --2- 172.21.15.7:0/3290790919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda406a930 0x7fdda415d0d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdda406e0b0 tx=0x7fdd94016000 comp rx=0 tx=0).stop 2023-10-07T07:57:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f35b2821700 1 -- 172.21.15.7:0/4181296119 wait complete. 2023-10-07T07:57:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.800+0000 7f35b2821700 1 Processor -- start 2023-10-07T07:57:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.800+0000 7fdda37fe700 1 -- 172.21.15.7:0/3290790919 reap_dead start 2023-10-07T07:57:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.800+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 shutdown_connections 2023-10-07T07:57:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.800+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 >> 172.21.15.7:0/3290790919 conn(0x7fdda4068bd0 msgr2=0x7fdda4144940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.800+0000 7f35b2821700 1 -- start start 2023-10-07T07:57:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35b2821700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35ac06e360 0x7f35ac0adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35b2821700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0cab80 0x7f35ac175c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35b2821700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b3e0 0x7f35ac1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35b2821700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0ac2c0 con 0x7f35ac15b3e0 2023-10-07T07:57:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35b2821700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0ab6f0 con 0x7f35ac0cab80 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35b2821700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0ab990 con 0x7f35ac06e360 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35b0dbe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b3e0 0x7f35ac1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35b0dbe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b3e0 0x7f35ac1579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33718/0 (socket says 172.21.15.7:33718) 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35abfff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35ac06e360 0x7f35ac0adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35b0dbe700 1 -- 172.21.15.7:0/1934023213 learned_addr learned my addr 172.21.15.7:0/1934023213 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.801+0000 7f35ab7fe700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0cab80 0x7f35ac175c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.802+0000 7f35ab7fe700 1 -- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35ac06e360 msgr2=0x7f35ac0adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.802+0000 7f35ab7fe700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35ac06e360 0x7f35ac0adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.802+0000 7f35ab7fe700 1 -- 172.21.15.7:0/1934023213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b3e0 msgr2=0x7f35ac1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.802+0000 7f35ab7fe700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b3e0 0x7f35ac1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.802+0000 7f35ab7fe700 1 -- 172.21.15.7:0/1934023213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a0009420 con 0x7f35ac0cab80 2023-10-07T07:57:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.802+0000 7f35ab7fe700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0cab80 0x7f35ac175c10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f35a0016000 tx=0x7f35a0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.802+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 shutdown_connections 2023-10-07T07:57:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.803+0000 7fdda95e7700 1 -- 172.21.15.7:0/3290790919 wait complete. 2023-10-07T07:57:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.798+0000 7f020b7fe700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f01ec066f30 0x7f01ec0693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f020c0af080 tx=0x7f020400d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.803+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 --> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8fbc002d30 con 0x7f8fc406f260 2023-10-07T07:57:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.803+0000 7f3593fff700 1 -- 172.21.15.7:0/1934023213 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a001f070 con 0x7f35ac0cab80 2023-10-07T07:57:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.803+0000 7f3593fff700 1 -- 172.21.15.7:0/1934023213 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a0000d10 con 0x7f35ac0cab80 2023-10-07T07:57:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.803+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac0abc70 con 0x7f35ac0cab80 2023-10-07T07:57:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.803+0000 7f8fd57fa700 1 -- 172.21.15.7:0/762594750 <== osd.5 v2:172.21.15.172:6808/120244 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8fbc002d30 con 0x7f8fc406f260 2023-10-07T07:57:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.803+0000 7f3593fff700 1 -- 172.21.15.7:0/1934023213 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000bc00 con 0x7f35ac0cab80 2023-10-07T07:57:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.803+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac0ae510 con 0x7f35ac0cab80 2023-10-07T07:57:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.803+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f8fc406f260 msgr2=0x7f8fc4071670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.804+0000 7f8fe78bb700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f8fc406f260 0x7f8fc4071670 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.804+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8fc4067050 msgr2=0x7f8fc4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.804+0000 7f8fe78bb700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8fc4067050 0x7f8fc4069500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006fd0 tx=0x7f8fdc008040 comp rx=0 tx=0).stop 2023-10-07T07:57:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.804+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe00c6be0 msgr2=0x7f8fe00ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.804+0000 7f8fe78bb700 1 --2- 172.21.15.7:0/762594750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe00c6be0 0x7f8fe00ae0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000b600 tx=0x7f8fd00091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.804+0000 7f8fe5e58700 1 -- 172.21.15.7:0/762594750 reap_dead start 2023-10-07T07:57:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.804+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35ac0ca670 con 0x7f35ac0cab80 2023-10-07T07:57:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.805+0000 7f3593fff700 1 -- 172.21.15.7:0/1934023213 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35a0023070 con 0x7f35ac0cab80 2023-10-07T07:57:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.805+0000 7f86fd7fa700 1 -- 172.21.15.7:0/1573259791 <== osd.0 v2:172.21.15.7:6809/120000 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f870006c340 con 0x7f86e406f090 2023-10-07T07:57:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.805+0000 7f3593fff700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3598067080 0x7f3598069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.805+0000 7f3593fff700 1 -- 172.21.15.7:0/1934023213 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f35a00859f0 con 0x7f35ac0cab80 2023-10-07T07:57:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.805+0000 7fd1667fc700 1 -- 172.21.15.7:0/2788934275 <== osd.3 v2:172.21.15.7:6800/120001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd17806f3f0 con 0x7fd15806f290 2023-10-07T07:57:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 shutdown_connections 2023-10-07T07:57:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 >> 172.21.15.7:0/762594750 conn(0x7f8fe0067a80 msgr2=0x7f8fe00c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f86e406f090 msgr2=0x7f86e40714a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f8707697700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f86e406f090 0x7f86e40714a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f86e4066f00 msgr2=0x7f86e40693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f8707697700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f86e4066f00 0x7f86e40693b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007580 tx=0x7f86f4009780 comp rx=0 tx=0).stop 2023-10-07T07:57:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 msgr2=0x7f87000afd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f8707697700 1 --2- 172.21.15.7:0/1573259791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8700155f20 0x7f87000afd50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0090b0 tx=0x7f86ec016000 comp rx=0 tx=0).stop 2023-10-07T07:57:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.807+0000 7f8705c34700 1 -- 172.21.15.7:0/1573259791 reap_dead start 2023-10-07T07:57:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.807+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 shutdown_connections 2023-10-07T07:57:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.807+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 >> 172.21.15.7:0/1573259791 conn(0x7f8700068bd0 msgr2=0x7f8700067880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.805+0000 7f3593fff700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7f359806f210 0x7f3598071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.805+0000 7f3593fff700 1 -- 172.21.15.7:0/1934023213 --> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3598071cd0 con 0x7f359806f210 2023-10-07T07:57:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.805+0000 7f3593fff700 1 -- 172.21.15.7:0/1934023213 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a0052c40 con 0x7f35ac0cab80 2023-10-07T07:57:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f35abfff700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3598067080 0x7f3598069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.806+0000 7f35abfff700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3598067080 0x7f3598069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f359c007130 tx=0x7f359c007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.807+0000 7f35b0dbe700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7f359806f210 0x7f3598071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.807+0000 7f35b0dbe700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7f359806f210 0x7f3598071620 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.808+0000 7f3593fff700 1 -- 172.21.15.7:0/1934023213 <== osd.6 v2:172.21.15.172:6816/120273 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3598071cd0 con 0x7f359806f210 2023-10-07T07:57:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.812+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 shutdown_connections 2023-10-07T07:57:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.812+0000 7f8fe78bb700 1 -- 172.21.15.7:0/762594750 wait complete. 2023-10-07T07:57:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.812+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 shutdown_connections 2023-10-07T07:57:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.812+0000 7f8707697700 1 -- 172.21.15.7:0/1573259791 wait complete. 2023-10-07T07:57:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.814+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7fd15806f290 msgr2=0x7fd1580716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.814+0000 7fd17c994700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7fd15806f290 0x7fd1580716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.814+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd158067080 msgr2=0x7fd158069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.814+0000 7fd17c994700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd158067080 0x7fd158069530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd178067900 tx=0x7fd168016000 comp rx=0 tx=0).stop 2023-10-07T07:57:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.814+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd17814e420 msgr2=0x7fd178154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.814+0000 7fd17c994700 1 --2- 172.21.15.7:0/2788934275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd17814e420 0x7fd178154cb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b4c0 tx=0x7fd16c009060 comp rx=0 tx=0).stop 2023-10-07T07:57:56.851 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:56.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-07T07:57:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.814+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 --> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f020c0716c0 con 0x7f01ec06f140 2023-10-07T07:57:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.814+0000 7f0203fff700 1 -- 172.21.15.7:0/1463376586 <== osd.7 v2:172.21.15.172:6824/120274 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f020c0716c0 con 0x7f01ec06f140 2023-10-07T07:57:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.823+0000 7fd176d9d700 1 -- 172.21.15.7:0/2788934275 reap_dead start 2023-10-07T07:57:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.823+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 shutdown_connections 2023-10-07T07:57:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.823+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 >> 172.21.15.7:0/2788934275 conn(0x7fd178067be0 msgr2=0x7fd17815d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.824+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f01ec06f140 msgr2=0x7f01ec071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.824+0000 7f0212795700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f01ec06f140 0x7f01ec071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.824+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f01ec066f30 msgr2=0x7f01ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.824+0000 7f0212795700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f01ec066f30 0x7f01ec0693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f020c0af080 tx=0x7f020400d040 comp rx=0 tx=0).stop 2023-10-07T07:57:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.824+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 msgr2=0x7f020c14b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.824+0000 7f0212795700 1 --2- 172.21.15.7:0/1463376586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f020c0c8cd0 0x7f020c14b2a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b610 tx=0x7f01fc009120 comp rx=0 tx=0).stop 2023-10-07T07:57:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.824+0000 7f0210d32700 1 -- 172.21.15.7:0/1463376586 reap_dead start 2023-10-07T07:57:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.825+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 shutdown_connections 2023-10-07T07:57:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.825+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 >> 172.21.15.7:0/1463376586 conn(0x7f020c0cd1a0 msgr2=0x7f020c0c9170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.854 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:56.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-07T07:57:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.825+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 shutdown_connections 2023-10-07T07:57:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.825+0000 7fd17c994700 1 -- 172.21.15.7:0/2788934275 wait complete. 2023-10-07T07:57:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.825+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 shutdown_connections 2023-10-07T07:57:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.825+0000 7f0212795700 1 -- 172.21.15.7:0/1463376586 wait complete. 2023-10-07T07:57:56.856 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:56.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-07T07:57:56.857 INFO:teuthology.orchestra.run.smithi007.stdout:55834574852 2023-10-07T07:57:56.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-07T07:57:56.859 INFO:teuthology.orchestra.run.smithi007.stdout:55834574852 2023-10-07T07:57:56.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-07T07:57:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.837+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 --> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35ac0716c0 con 0x7f359806f210 2023-10-07T07:57:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.837+0000 7f3593fff700 1 -- 172.21.15.7:0/1934023213 <== osd.6 v2:172.21.15.172:6816/120273 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f35ac0716c0 con 0x7f359806f210 2023-10-07T07:57:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.837+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7f359806f210 msgr2=0x7f3598071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.837+0000 7f35b2821700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7f359806f210 0x7f3598071620 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.838+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3598067080 msgr2=0x7f3598069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.838+0000 7f35b2821700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3598067080 0x7f3598069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f359c007130 tx=0x7f359c007eb0 comp rx=0 tx=0).stop 2023-10-07T07:57:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.838+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0cab80 msgr2=0x7f35ac175c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.838+0000 7f35b2821700 1 --2- 172.21.15.7:0/1934023213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0cab80 0x7f35ac175c10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f35a0016000 tx=0x7f35a0017000 comp rx=0 tx=0).stop 2023-10-07T07:57:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.838+0000 7f35b0dbe700 1 -- 172.21.15.7:0/1934023213 reap_dead start 2023-10-07T07:57:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.838+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 shutdown_connections 2023-10-07T07:57:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.838+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 >> 172.21.15.7:0/1934023213 conn(0x7f35ac0cbe90 msgr2=0x7f35ac0c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.838+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 shutdown_connections 2023-10-07T07:57:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:56.838+0000 7f35b2821700 1 -- 172.21.15.7:0/1934023213 wait complete. 2023-10-07T07:57:56.862 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-10-07T07:57:56.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-07T07:57:56.863 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:56.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-07T07:57:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:56.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-07T07:57:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.131+0000 7ff64b901700 1 Processor -- start 2023-10-07T07:57:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.131+0000 7ff64b901700 1 -- start start 2023-10-07T07:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.132+0000 7ff64b901700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff644159380 0x7ff6440b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff64b901700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6440b4270 0x7ff6440d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff64b901700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6440d0be0 con 0x7ff6440d0250 2023-10-07T07:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff64b901700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6440b4820 con 0x7ff6440b4270 2023-10-07T07:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff64b901700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440b5000 con 0x7ff644159380 2023-10-07T07:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff2c40d5700 1 Processor -- start 2023-10-07T07:57:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff648e9c700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff6440d0250 0x7ff6440b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:60010/0 (socket says 172.21.15.7:60010) 2023-10-07T07:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff648e9c700 1 -- 172.21.15.7:0/3977162140 learned_addr learned my addr 172.21.15.7:0/3977162140 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff64969d700 1 --2- 172.21.15.7:0/3977162140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff644159380 0x7ff6440b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff649e9e700 1 --2- 172.21.15.7:0/3977162140 >> v2:172.21.15.7:3300/0 conn(0x7ff6440b4270 0x7ff6440d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff63bfff700 1 -- 172.21.15.7:0/3977162140 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328173101 0 0) 0x7ff6440d0be0 con 0x7ff6440d0250 2023-10-07T07:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff63bfff700 1 -- 172.21.15.7:0/3977162140 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff630003570 con 0x7ff6440d0250 2023-10-07T07:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff64969d700 1 -- 172.21.15.7:0/3977162140 >> v2:172.21.15.7:3300/0 conn(0x7ff6440b4270 msgr2=0x7ff6440d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff64969d700 1 --2- 172.21.15.7:0/3977162140 >> v2:172.21.15.7:3300/0 conn(0x7ff6440b4270 0x7ff6440d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff64969d700 1 -- 172.21.15.7:0/3977162140 >> v1:172.21.15.172:6789/0 conn(0x7ff6440d0250 legacy=0x7ff6440b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff64969d700 1 -- 172.21.15.7:0/3977162140 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6440b52e0 con 0x7ff644159380 2023-10-07T07:57:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff64969d700 1 --2- 172.21.15.7:0/3977162140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff644159380 0x7ff6440b5ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009e60 tx=0x7ff63c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b798346874d36938 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.133+0000 7ff2c40d5700 1 -- start start 2023-10-07T07:57:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff2c40d5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0cb3c0 0x7ff2bc0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff2c40d5700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff2bc0c64d0 0x7ff2bc0c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff2c40d5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff2bc15e2b0 0x7ff2bc15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff2c40d5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2bc140cd0 con 0x7ff2bc15e2b0 2023-10-07T07:57:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff2c40d5700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2bc0c6e80 con 0x7ff2bc0c64d0 2023-10-07T07:57:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7ff2c40d5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0c9a60 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff63bfff700 1 -- 172.21.15.7:0/3977162140 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00e070 con 0x7ff644159380 2023-10-07T07:57:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff63bfff700 1 -- 172.21.15.7:0/3977162140 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c00bb60 con 0x7ff644159380 2023-10-07T07:57:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff2c1670700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff2bc0c64d0 0x7ff2bc0c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff2c1670700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff2bc0c64d0 0x7ff2bc0c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39442/0 (socket says 172.21.15.7:39442) 2023-10-07T07:57:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff2c1670700 1 -- 172.21.15.7:0/2858136931 learned_addr learned my addr 172.21.15.7:0/2858136931 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff2c2672700 1 --2- 172.21.15.7:0/2858136931 >> v2:172.21.15.7:3300/0 conn(0x7ff2bc15e2b0 0x7ff2bc15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff63bfff700 1 -- 172.21.15.7:0/3977162140 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c012430 con 0x7ff644159380 2023-10-07T07:57:57.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7fdcdd12b700 1 Processor -- start 2023-10-07T07:57:57.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.134+0000 7fdcdd12b700 1 -- start start 2023-10-07T07:57:57.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7fdcdd12b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdcd8157470 0x7fdcd806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7fdcdd12b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd80c8f10 con 0x7fdcd81570a0 2023-10-07T07:57:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7fdcdd12b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd80c3b80 con 0x7fdcd806bb60 2023-10-07T07:57:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7fdcdd12b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd806bfe0 con 0x7fdcd8157470 2023-10-07T07:57:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff2c2672700 1 -- 172.21.15.7:0/2858136931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0cb3c0 msgr2=0x7ff2bc0cb790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff2c2672700 1 --2- 172.21.15.7:0/2858136931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0cb3c0 0x7ff2bc0cb790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff2c2672700 1 -- 172.21.15.7:0/2858136931 >> v2:172.21.15.172:3300/0 conn(0x7ff2bc0c64d0 msgr2=0x7ff2bc0c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff2c2672700 1 --2- 172.21.15.7:0/2858136931 >> v2:172.21.15.172:3300/0 conn(0x7ff2bc0c64d0 0x7ff2bc0c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7ff2c2672700 1 -- 172.21.15.7:0/2858136931 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc0cbcd0 con 0x7ff2bc15e2b0 2023-10-07T07:57:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7fdcd6d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdcd81570a0 0x7fdcd80c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36008/0 (socket says 172.21.15.7:36008) 2023-10-07T07:57:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.135+0000 7fdcd6d9d700 1 -- 172.21.15.7:0/3709975935 learned_addr learned my addr 172.21.15.7:0/3709975935 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7fdcd659c700 1 --2- 172.21.15.7:0/3709975935 >> v2:172.21.15.172:3300/0 conn(0x7fdcd8157470 0x7fdcd806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7fdcd5d9b700 1 -- 172.21.15.7:0/3709975935 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631003011 0 0) 0x7fdcd0001490 con 0x7fdcd81570a0 2023-10-07T07:57:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7fdcd5d9b700 1 -- 172.21.15.7:0/3709975935 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc0003570 con 0x7fdcd81570a0 2023-10-07T07:57:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7ff64b901700 1 -- 172.21.15.7:0/3977162140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff644159380 msgr2=0x7ff6440b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7ff64b901700 1 --2- 172.21.15.7:0/3977162140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff644159380 0x7ff6440b5ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009e60 tx=0x7ff63c009780 comp rx=0 tx=0).stop 2023-10-07T07:57:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7fdcd659c700 1 -- 172.21.15.7:0/3709975935 >> v1:172.21.15.172:6789/0 conn(0x7fdcd806bb60 legacy=0x7fdcd80c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7fdcd659c700 1 -- 172.21.15.7:0/3709975935 >> v1:172.21.15.7:6789/0 conn(0x7fdcd81570a0 legacy=0x7fdcd80c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7fdcd659c700 1 -- 172.21.15.7:0/3709975935 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd80c2f70 con 0x7fdcd8157470 2023-10-07T07:57:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7ff2c2672700 1 --2- 172.21.15.7:0/2858136931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc15e2b0 0x7ff2bc15e6d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc06ae80 tx=0x7ff2b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7184e1c5d1d632 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7fdcd659c700 1 --2- 172.21.15.7:0/3709975935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcd8157470 0x7fdcd806b620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009860 tx=0x7fdcc80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e88f62ea991457f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7ff64b901700 1 -- 172.21.15.7:0/3977162140 shutdown_connections 2023-10-07T07:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7ff64b901700 1 --2- 172.21.15.7:0/3977162140 >> v2:172.21.15.7:3300/0 conn(0x7ff6440b4270 0x7ff6440d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7ff64b901700 1 --2- 172.21.15.7:0/3977162140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff644159380 0x7ff6440b5ae0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.136+0000 7ff64b901700 1 -- 172.21.15.7:0/3977162140 >> 172.21.15.7:0/3977162140 conn(0x7ff644067be0 msgr2=0x7ff644067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.137+0000 7fdcd5d9b700 1 -- 172.21.15.7:0/3709975935 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8011630 con 0x7fdcd8157470 2023-10-07T07:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.137+0000 7fdcd5d9b700 1 -- 172.21.15.7:0/3709975935 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc8011790 con 0x7fdcd8157470 2023-10-07T07:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.137+0000 7fdcd5d9b700 1 -- 172.21.15.7:0/3709975935 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8011b30 con 0x7fdcd8157470 2023-10-07T07:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.137+0000 7ff64b901700 1 -- 172.21.15.7:0/3977162140 shutdown_connections 2023-10-07T07:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.137+0000 7ff64b901700 1 -- 172.21.15.7:0/3977162140 wait complete. 2023-10-07T07:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7fdcdd12b700 1 -- 172.21.15.7:0/3709975935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcd8157470 msgr2=0x7fdcd806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7fdcdd12b700 1 --2- 172.21.15.7:0/3709975935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcd8157470 0x7fdcd806b620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009860 tx=0x7fdcc80091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7fdcdd12b700 1 -- 172.21.15.7:0/3709975935 shutdown_connections 2023-10-07T07:57:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7fdcdd12b700 1 --2- 172.21.15.7:0/3709975935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcd8157470 0x7fdcd806b620 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7fdcdd12b700 1 -- 172.21.15.7:0/3709975935 >> 172.21.15.7:0/3709975935 conn(0x7fdcd8067ac0 msgr2=0x7fdcd80ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7f341b82d700 1 Processor -- start 2023-10-07T07:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7fdcdd12b700 1 -- 172.21.15.7:0/3709975935 shutdown_connections 2023-10-07T07:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7fdcdd12b700 1 -- 172.21.15.7:0/3709975935 wait complete. 2023-10-07T07:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7ff64b901700 1 Processor -- start 2023-10-07T07:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7ff2c0e6f700 1 -- 172.21.15.7:0/2858136931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400e070 con 0x7ff2bc15e2b0 2023-10-07T07:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7ff2c0e6f700 1 -- 172.21.15.7:0/2858136931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b400bb60 con 0x7ff2bc15e2b0 2023-10-07T07:57:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7ff2c0e6f700 1 -- 172.21.15.7:0/2858136931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4012430 con 0x7ff2bc15e2b0 2023-10-07T07:57:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.139+0000 7ff2c40d5700 1 -- 172.21.15.7:0/2858136931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc15e2b0 msgr2=0x7ff2bc15e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.139+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/2858136931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc15e2b0 0x7ff2bc15e6d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc06ae80 tx=0x7ff2b4009780 comp rx=0 tx=0).stop 2023-10-07T07:57:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7ff64b901700 1 -- start start 2023-10-07T07:57:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff2c40d5700 1 -- 172.21.15.7:0/2858136931 shutdown_connections 2023-10-07T07:57:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/2858136931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc15e2b0 0x7ff2bc15e6d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/2858136931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0cb3c0 0x7ff2bc0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/2858136931 >> v2:172.21.15.172:3300/0 conn(0x7ff2bc0c64d0 0x7ff2bc0c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff2c40d5700 1 -- 172.21.15.7:0/2858136931 >> 172.21.15.7:0/2858136931 conn(0x7ff2bc067a80 msgr2=0x7ff2bc0cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7fdcdd12b700 1 Processor -- start 2023-10-07T07:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.139+0000 7f7a8a7e7700 1 Processor -- start 2023-10-07T07:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f7a8a7e7700 1 -- start start 2023-10-07T07:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.138+0000 7f341b82d700 1 -- start start 2023-10-07T07:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f341b82d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34140cb3c0 0x7f34140cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f341b82d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34140c9a60 con 0x7f34140c64d0 2023-10-07T07:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f341b82d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34140cbcd0 con 0x7f341415e9c0 2023-10-07T07:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f341b82d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3414140cd0 con 0x7f34140cb3c0 2023-10-07T07:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff64b901700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6440b4270 0x7ff644158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff64b901700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6440d0250 0x7ff644157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff64b901700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff644159380 0x7ff644179ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff64b901700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64415cce0 con 0x7ff6440d0250 2023-10-07T07:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff64b901700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64406a3b0 con 0x7ff644159380 2023-10-07T07:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7ff64b901700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64406a650 con 0x7ff6440b4270 2023-10-07T07:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f34195c9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34140cb3c0 0x7f34140cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f34195c9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34140cb3c0 0x7f34140cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33740/0 (socket says 172.21.15.7:33740) 2023-10-07T07:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f34195c9700 1 -- 172.21.15.7:0/2983481755 learned_addr learned my addr 172.21.15.7:0/2983481755 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f9f9a54a700 1 Processor -- start 2023-10-07T07:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7f9f9a54a700 1 -- start start 2023-10-07T07:57:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff648e9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6440d0250 0x7ff644157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff648e9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6440d0250 0x7ff644157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33742/0 (socket says 172.21.15.7:33742) 2023-10-07T07:57:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff648e9c700 1 -- 172.21.15.7:0/2420005114 learned_addr learned my addr 172.21.15.7:0/2420005114 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff64969d700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6440b4270 0x7ff644158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff649e9e700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff644159380 0x7ff644179ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f7a8a7e7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7a8413b920 0x7f7a840c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f7a8a7e7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7a840c6be0 0x7f7a840695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f7a8a7e7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a84156900 con 0x7f7a84156460 2023-10-07T07:57:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f7a8a7e7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a840cb040 con 0x7f7a8413b920 2023-10-07T07:57:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f7a8a7e7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a840ca430 con 0x7f7a840c6be0 2023-10-07T07:57:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f34195c9700 1 -- 172.21.15.7:0/2983481755 >> v1:172.21.15.172:6789/0 conn(0x7f341415e9c0 legacy=0x7f3414069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f34195c9700 1 -- 172.21.15.7:0/2983481755 >> v1:172.21.15.7:6789/0 conn(0x7f34140c64d0 legacy=0x7f341415e2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T07:57:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f34195c9700 1 -- 172.21.15.7:0/2983481755 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341406a240 con 0x7f34140cb3c0 2023-10-07T07:57:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.140+0000 7fdcdd12b700 1 -- start start 2023-10-07T07:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7fdcdd12b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcd806bb60 0x7fdcd8139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7fdcdd12b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd81570a0 0x7fdcd80b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7fdcdd12b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcd8157470 0x7fdcd813cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7fdcdd12b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd81766c0 con 0x7fdcd81570a0 2023-10-07T07:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7fdcdd12b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd813ac60 con 0x7fdcd806bb60 2023-10-07T07:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7fdcdd12b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd813a930 con 0x7fdcd8157470 2023-10-07T07:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff64969d700 1 -- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff644159380 msgr2=0x7ff644179ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff64969d700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff644159380 0x7ff644179ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff64969d700 1 -- 172.21.15.7:0/2420005114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6440d0250 msgr2=0x7ff644157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff64969d700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6440d0250 0x7ff644157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff64969d700 1 -- 172.21.15.7:0/2420005114 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c009a20 con 0x7ff6440b4270 2023-10-07T07:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7ff648e9c700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6440d0250 0x7ff644157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f7a88d84700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7a840c6be0 0x7f7a840695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.141+0000 7f7a88d84700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7a840c6be0 0x7f7a840695c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39464/0 (socket says 172.21.15.7:39464) 2023-10-07T07:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f7a88d84700 1 -- 172.21.15.7:0/817703687 learned_addr learned my addr 172.21.15.7:0/817703687 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7ff2c40d5700 1 -- 172.21.15.7:0/2858136931 shutdown_connections 2023-10-07T07:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f9f9a54a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 0x7f9f94161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f9f9a54a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9f940b4760 0x7f9f9415d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f9f9a54a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f940b57e0 con 0x7f9f9406f280 2023-10-07T07:57:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f9f9a54a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f940ccd20 con 0x7f9f940b4760 2023-10-07T07:57:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f9f9a54a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940b4f40 con 0x7f9f9406eeb0 2023-10-07T07:57:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7fdcd759e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcd8157470 0x7fdcd813cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7fdcd759e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcd8157470 0x7fdcd813cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:60888/0 (socket says 172.21.15.7:60888) 2023-10-07T07:57:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7fdcd759e700 1 -- 172.21.15.7:0/2936500964 learned_addr learned my addr 172.21.15.7:0/2936500964 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7ff64969d700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6440b4270 0x7ff644158280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff63c016000 tx=0x7ff63c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f9f937fe700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9f9406f280 0x7f9f940cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:60040/0 (socket says 172.21.15.7:60040) 2023-10-07T07:57:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f9f937fe700 1 -- 172.21.15.7:0/506987077 learned_addr learned my addr 172.21.15.7:0/506987077 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f9f93fff700 1 --2- 172.21.15.7:0/506987077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 0x7f9f94161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7ff6397fa700 1 -- 172.21.15.7:0/2420005114 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c012cf0 con 0x7ff6440b4270 2023-10-07T07:57:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7ff6397fa700 1 -- 172.21.15.7:0/2420005114 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c027470 con 0x7ff6440b4270 2023-10-07T07:57:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7ff6397fa700 1 -- 172.21.15.7:0/2420005114 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c027830 con 0x7ff6440b4270 2023-10-07T07:57:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f7a83fff700 1 --2- 172.21.15.7:0/817703687 >> v2:172.21.15.7:3300/0 conn(0x7f7a8413b920 0x7f7a840c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f7a88d84700 1 -- 172.21.15.7:0/817703687 >> v2:172.21.15.7:3300/0 conn(0x7f7a8413b920 msgr2=0x7f7a840c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f7a88d84700 1 --2- 172.21.15.7:0/817703687 >> v2:172.21.15.7:3300/0 conn(0x7f7a8413b920 0x7f7a840c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f7a88d84700 1 -- 172.21.15.7:0/817703687 >> v1:172.21.15.7:6789/0 conn(0x7f7a84156460 legacy=0x7f7a840c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7f7a88d84700 1 -- 172.21.15.7:0/817703687 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840ca710 con 0x7f7a840c6be0 2023-10-07T07:57:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f7a88d84700 1 --2- 172.21.15.7:0/817703687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a840c6be0 0x7f7a840695c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7a840cb3e0 tx=0x7f7a7c007690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9775a4cefd39c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.142+0000 7ff2c40d5700 1 -- 172.21.15.7:0/2858136931 wait complete. 2023-10-07T07:57:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f34195c9700 1 --2- 172.21.15.7:0/2983481755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34140cb3c0 0x7f34140cb790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34040099b0 tx=0x7f3404009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1098725bce967545 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f9f93fff700 1 -- 172.21.15.7:0/506987077 >> v2:172.21.15.7:3300/0 conn(0x7f9f940b4760 msgr2=0x7f9f9415d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f9f93fff700 1 --2- 172.21.15.7:0/506987077 >> v2:172.21.15.7:3300/0 conn(0x7f9f940b4760 0x7f9f9415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f9f93fff700 1 -- 172.21.15.7:0/506987077 >> v1:172.21.15.172:6789/0 conn(0x7f9f9406f280 legacy=0x7f9f940cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f9f93fff700 1 -- 172.21.15.7:0/506987077 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f940b5220 con 0x7f9f9406eeb0 2023-10-07T07:57:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f7a82ffd700 1 -- 172.21.15.7:0/817703687 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00de50 con 0x7f7a840c6be0 2023-10-07T07:57:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f7a82ffd700 1 -- 172.21.15.7:0/817703687 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c007b40 con 0x7f7a840c6be0 2023-10-07T07:57:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f7a82ffd700 1 -- 172.21.15.7:0/817703687 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0124d0 con 0x7f7a840c6be0 2023-10-07T07:57:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f9f93fff700 1 --2- 172.21.15.7:0/506987077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 0x7f9f94161f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f840098e0 tx=0x7f9f840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9262c7fcdd1cc3b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f340bfff700 1 -- 172.21.15.7:0/2983481755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400b7e0 con 0x7f34140cb3c0 2023-10-07T07:57:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f340bfff700 1 -- 172.21.15.7:0/2983481755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340400b940 con 0x7f34140cb3c0 2023-10-07T07:57:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f340bfff700 1 -- 172.21.15.7:0/2983481755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400bce0 con 0x7f34140cb3c0 2023-10-07T07:57:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f9f92ffd700 1 -- 172.21.15.7:0/506987077 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84009e80 con 0x7f9f9406eeb0 2023-10-07T07:57:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f9f92ffd700 1 -- 172.21.15.7:0/506987077 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8400b820 con 0x7f9f9406eeb0 2023-10-07T07:57:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7f9f92ffd700 1 -- 172.21.15.7:0/506987077 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400bbe0 con 0x7f9f9406eeb0 2023-10-07T07:57:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7fdcd659c700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd81570a0 0x7fdcd80b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7f4726e43700 1 Processor -- start 2023-10-07T07:57:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64406a930 con 0x7ff6440b4270 2023-10-07T07:57:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6441795b0 con 0x7ff6440b4270 2023-10-07T07:57:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7f341b82d700 1 -- 172.21.15.7:0/2983481755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34140cb3c0 msgr2=0x7f34140cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7f341b82d700 1 --2- 172.21.15.7:0/2983481755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34140cb3c0 0x7f34140cb790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34040099b0 tx=0x7f3404009250 comp rx=0 tx=0).stop 2023-10-07T07:57:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7fdcd659c700 1 -- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcd8157470 msgr2=0x7fdcd813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7fdcd659c700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcd8157470 0x7fdcd813cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7fdcd659c700 1 -- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcd806bb60 msgr2=0x7fdcd8139930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-07T07:57:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7fdcd659c700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcd806bb60 0x7fdcd8139930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7fdcd659c700 1 -- 172.21.15.7:0/2936500964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc8009420 con 0x7fdcd81570a0 2023-10-07T07:57:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7f9f9a54a700 1 -- 172.21.15.7:0/506987077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 msgr2=0x7f9f94161f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7f9f9a54a700 1 --2- 172.21.15.7:0/506987077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 0x7f9f94161f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f840098e0 tx=0x7f9f840091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7fdcd659c700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd81570a0 0x7fdcd80b1800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8015000 tx=0x7fdcc8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7fdcbeffd700 1 -- 172.21.15.7:0/2936500964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc801cd30 con 0x7fdcd81570a0 2023-10-07T07:57:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.144+0000 7f4726e43700 1 -- start start 2023-10-07T07:57:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f4726e43700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47200cb3c0 0x7f47200cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f4726e43700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f47200c64d0 0x7f47200c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f4726e43700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472015e2b0 0x7f472015e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f4726e43700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4720140cd0 con 0x7f47200cb3c0 2023-10-07T07:57:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f4726e43700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47200c6e90 con 0x7f47200c64d0 2023-10-07T07:57:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f4726e43700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47200c9a60 con 0x7f472015e2b0 2023-10-07T07:57:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f341b82d700 1 -- 172.21.15.7:0/2983481755 shutdown_connections 2023-10-07T07:57:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f341b82d700 1 --2- 172.21.15.7:0/2983481755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34140cb3c0 0x7f34140cb790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f341b82d700 1 -- 172.21.15.7:0/2983481755 >> 172.21.15.7:0/2983481755 conn(0x7f3414067a80 msgr2=0x7f34140cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7fdcbeffd700 1 -- 172.21.15.7:0/2936500964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc801ce90 con 0x7fdcd81570a0 2023-10-07T07:57:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd80b21c0 con 0x7fdcd81570a0 2023-10-07T07:57:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7fdcbeffd700 1 -- 172.21.15.7:0/2936500964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc801a680 con 0x7fdcd81570a0 2023-10-07T07:57:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd80b2780 con 0x7fdcd81570a0 2023-10-07T07:57:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f47253e0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472015e2b0 0x7f472015e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.143+0000 7ff2c40d5700 1 Processor -- start 2023-10-07T07:57:57.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/817703687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a840c6be0 msgr2=0x7f7a840695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.145+0000 7f7a8a7e7700 1 --2- 172.21.15.7:0/817703687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a840c6be0 0x7f7a840695c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7a840cb3e0 tx=0x7f7a7c007690 comp rx=0 tx=0).stop 2023-10-07T07:57:57.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7f4724bdf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47200cb3c0 0x7f47200cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7f4724bdf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47200cb3c0 0x7f47200cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33780/0 (socket says 172.21.15.7:33780) 2023-10-07T07:57:57.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7f4724bdf700 1 -- 172.21.15.7:0/4275061584 learned_addr learned my addr 172.21.15.7:0/4275061584 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7f4724bdf700 1 -- 172.21.15.7:0/4275061584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472015e2b0 msgr2=0x7f472015e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7f4724bdf700 1 --2- 172.21.15.7:0/4275061584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472015e2b0 0x7f472015e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7f4724bdf700 1 -- 172.21.15.7:0/4275061584 >> v2:172.21.15.172:3300/0 conn(0x7f47200c64d0 msgr2=0x7f47200c6950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T07:57:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7f4724bdf700 1 --2- 172.21.15.7:0/4275061584 >> v2:172.21.15.172:3300/0 conn(0x7f47200c64d0 0x7f47200c6950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7f4724bdf700 1 -- 172.21.15.7:0/4275061584 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47200cbcd0 con 0x7f47200cb3c0 2023-10-07T07:57:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7ff6397fa700 1 -- 172.21.15.7:0/2420005114 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff63c024070 con 0x7ff6440b4270 2023-10-07T07:57:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7f341b82d700 1 -- 172.21.15.7:0/2983481755 shutdown_connections 2023-10-07T07:57:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f4724bdf700 1 --2- 172.21.15.7:0/4275061584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47200cb3c0 0x7f47200cb790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f47201612e0 tx=0x7f471c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367524d02fb410ee server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.146+0000 7ff2c40d5700 1 -- start start 2023-10-07T07:57:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff2c40d5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0c64d0 0x7ff2bc156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff2c40d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc0cb3c0 0x7ff2bc1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff2c40d5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2bc15e2b0 0x7ff2bc153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff2c40d5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0ad1f0 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff2c40d5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0adc80 con 0x7ff2bc15e2b0 2023-10-07T07:57:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff2c40d5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc15bae0 con 0x7ff2bc0c64d0 2023-10-07T07:57:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f341b82d700 1 -- 172.21.15.7:0/2983481755 wait complete. 2023-10-07T07:57:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff6397fa700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff628067080 0x7ff628069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f47177fe700 1 -- 172.21.15.7:0/4275061584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00b820 con 0x7f47200cb3c0 2023-10-07T07:57:57.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f47177fe700 1 -- 172.21.15.7:0/4275061584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c00b980 con 0x7f47200cb3c0 2023-10-07T07:57:57.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f47177fe700 1 -- 172.21.15.7:0/4275061584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00bd20 con 0x7f47200cb3c0 2023-10-07T07:57:57.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff6397fa700 1 -- 172.21.15.7:0/2420005114 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff63c086a70 con 0x7ff6440b4270 2023-10-07T07:57:57.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6441797a0 con 0x7ff6440b4270 2023-10-07T07:57:57.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff648e9c700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff628067080 0x7ff628069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/817703687 shutdown_connections 2023-10-07T07:57:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f7a8a7e7700 1 --2- 172.21.15.7:0/817703687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a840c6be0 0x7f7a840695c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f7a8a7e7700 1 --2- 172.21.15.7:0/817703687 >> v2:172.21.15.7:3300/0 conn(0x7f7a8413b920 0x7f7a840c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/817703687 >> 172.21.15.7:0/817703687 conn(0x7f7a84067a80 msgr2=0x7f7a840cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/817703687 shutdown_connections 2023-10-07T07:57:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff2c1e71700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0c64d0 0x7ff2bc156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7ff2c2672700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2bc15e2b0 0x7ff2bc153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff2c1e71700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0c64d0 0x7ff2bc156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:60912/0 (socket says 172.21.15.7:60912) 2023-10-07T07:57:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff2c1e71700 1 -- 172.21.15.7:0/1151251598 learned_addr learned my addr 172.21.15.7:0/1151251598 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff2c1670700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc0cb3c0 0x7ff2bc1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f9f9a54a700 1 -- 172.21.15.7:0/506987077 shutdown_connections 2023-10-07T07:57:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f9f9a54a700 1 --2- 172.21.15.7:0/506987077 >> v2:172.21.15.7:3300/0 conn(0x7f9f940b4760 0x7f9f9415d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f9f9a54a700 1 --2- 172.21.15.7:0/506987077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 0x7f9f94161f50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f9f9a54a700 1 -- 172.21.15.7:0/506987077 >> 172.21.15.7:0/506987077 conn(0x7f9f94067be0 msgr2=0x7f9f94067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff648e9c700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff628067080 0x7ff628069530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff644131ac0 tx=0x7ff640009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff2c1670700 1 -- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0c64d0 msgr2=0x7ff2bc156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff2c1670700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0c64d0 0x7ff2bc156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff2c1670700 1 -- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2bc15e2b0 msgr2=0x7ff2bc153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff2c1670700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2bc15e2b0 0x7ff2bc153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.148+0000 7ff2c1670700 1 -- 172.21.15.7:0/1151251598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b4009a20 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7ff2c1670700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc0cb3c0 0x7ff2bc1573e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8006d40 tx=0x7ff2b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.147+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/817703687 wait complete. 2023-10-07T07:57:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7f341b82d700 1 Processor -- start 2023-10-07T07:57:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7f9f9a54a700 1 -- 172.21.15.7:0/506987077 shutdown_connections 2023-10-07T07:57:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7f4726e43700 1 -- 172.21.15.7:0/4275061584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47200cb3c0 msgr2=0x7f47200cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7f4726e43700 1 --2- 172.21.15.7:0/4275061584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47200cb3c0 0x7f47200cb790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f47201612e0 tx=0x7f471c009250 comp rx=0 tx=0).stop 2023-10-07T07:57:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7ff2b1ffb700 1 -- 172.21.15.7:0/1151251598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8013680 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7ff2b1ffb700 1 -- 172.21.15.7:0/1151251598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b80137e0 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7ff2b1ffb700 1 -- 172.21.15.7:0/1151251598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8013b60 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc0ad3e0 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7f7a8a7e7700 1 Processor -- start 2023-10-07T07:57:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc0ad9a0 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7f9f9a54a700 1 -- 172.21.15.7:0/506987077 wait complete. 2023-10-07T07:57:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7fdcbeffd700 1 -- 172.21.15.7:0/2936500964 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcc801aa70 con 0x7fdcd81570a0 2023-10-07T07:57:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.150+0000 7fdcbeffd700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdcc4066fb0 0x7fdcc4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.150+0000 7fdcbeffd700 1 -- 172.21.15.7:0/2936500964 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdcc8085c00 con 0x7fdcd81570a0 2023-10-07T07:57:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.150+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2bc15bc70 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.150+0000 7f9f9a54a700 1 Processor -- start 2023-10-07T07:57:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb80052b0 con 0x7fdcd81570a0 2023-10-07T07:57:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.149+0000 7f341b82d700 1 -- start start 2023-10-07T07:57:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.150+0000 7f341b82d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34140c64d0 0x7f34140b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.150+0000 7f341b82d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34140cb3c0 0x7f34140aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.150+0000 7f341b82d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f341415e9c0 0x7f34140af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f341b82d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34140ac100 con 0x7f341415e9c0 2023-10-07T07:57:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f341b82d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34140ac350 con 0x7f34140c64d0 2023-10-07T07:57:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f341b82d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34140ac630 con 0x7f34140cb3c0 2023-10-07T07:57:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f34195c9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34140c64d0 0x7f34140b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f34195c9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34140c64d0 0x7f34140b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39482/0 (socket says 172.21.15.7:39482) 2023-10-07T07:57:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f34195c9700 1 -- 172.21.15.7:0/464170555 learned_addr learned my addr 172.21.15.7:0/464170555 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f34195c9700 1 -- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34140cb3c0 msgr2=0x7f34140aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f34195c9700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34140cb3c0 0x7f34140aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f34195c9700 1 -- 172.21.15.7:0/464170555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f341415e9c0 msgr2=0x7f34140af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f34195c9700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f341415e9c0 0x7f34140af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f34195c9700 1 -- 172.21.15.7:0/464170555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34040094f0 con 0x7f34140c64d0 2023-10-07T07:57:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.150+0000 7f7a8a7e7700 1 -- start start 2023-10-07T07:57:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a8a7e7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a840c6be0 0x7f7a840b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a8a7e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8413b920 0x7f7a840ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a8a7e7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a84156460 0x7f7a840b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a8a7e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84176f40 con 0x7f7a8413b920 2023-10-07T07:57:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a8a7e7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8413d5c0 con 0x7f7a84156460 2023-10-07T07:57:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a8a7e7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840b1360 con 0x7f7a840c6be0 2023-10-07T07:57:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a83fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a840c6be0 0x7f7a840b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a88d84700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a84156460 0x7f7a840b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a83fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a840c6be0 0x7f7a840b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:60934/0 (socket says 172.21.15.7:60934) 2023-10-07T07:57:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f7a83fff700 1 -- 172.21.15.7:0/1211273246 learned_addr learned my addr 172.21.15.7:0/1211273246 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f34195c9700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34140c64d0 0x7f34140b1550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f341406cc40 tx=0x7f3404016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f34097fa700 1 -- 172.21.15.7:0/464170555 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404011e10 con 0x7f34140c64d0 2023-10-07T07:57:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f7a88d84700 1 -- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a840c6be0 msgr2=0x7f7a840b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f7a88d84700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a840c6be0 0x7f7a840b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f7a88d84700 1 -- 172.21.15.7:0/1211273246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8413b920 msgr2=0x7f7a840ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f7a88d84700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8413b920 0x7f7a840ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f7a88d84700 1 -- 172.21.15.7:0/1211273246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c00be70 con 0x7f7a84156460 2023-10-07T07:57:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f34097fa700 1 -- 172.21.15.7:0/464170555 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3404020070 con 0x7f34140c64d0 2023-10-07T07:57:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f34097fa700 1 -- 172.21.15.7:0/464170555 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340401b6a0 con 0x7f34140c64d0 2023-10-07T07:57:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.151+0000 7f9f9a54a700 1 -- start start 2023-10-07T07:57:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f9a54a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 0x7f9f941436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f9a54a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f9406f280 0x7f9f9415f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f9a54a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f940b4760 0x7f9f9415f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f9a54a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9416ef50 con 0x7f9f9406f280 2023-10-07T07:57:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f9a54a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9416f8d0 con 0x7f9f940b4760 2023-10-07T07:57:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f9a54a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9416ec70 con 0x7f9f9406eeb0 2023-10-07T07:57:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34140ac910 con 0x7f34140c64d0 2023-10-07T07:57:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3414140570 con 0x7f34140c64d0 2023-10-07T07:57:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f7a88d84700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a84156460 0x7f7a840b0150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a8406f910 tx=0x7f7a7c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f98ae7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f940b4760 0x7f9f9415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f98ae7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f940b4760 0x7f9f9415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39494/0 (socket says 172.21.15.7:39494) 2023-10-07T07:57:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f98ae7700 1 -- 172.21.15.7:0/282761721 learned_addr learned my addr 172.21.15.7:0/282761721 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f98ae7700 1 -- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 msgr2=0x7f9f941436d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T07:57:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f98ae7700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 0x7f9f941436d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f98ae7700 1 -- 172.21.15.7:0/282761721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f9406f280 msgr2=0x7f9f9415f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f98ae7700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f9406f280 0x7f9f9415f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f9f98ae7700 1 -- 172.21.15.7:0/282761721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f84009420 con 0x7f9f940b4760 2023-10-07T07:57:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.152+0000 7f7a73fff700 1 -- 172.21.15.7:0/1211273246 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01c620 con 0x7f7a84156460 2023-10-07T07:57:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.153+0000 7f7a73fff700 1 -- 172.21.15.7:0/1211273246 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c01c7a0 con 0x7f7a84156460 2023-10-07T07:57:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.153+0000 7f7a73fff700 1 -- 172.21.15.7:0/1211273246 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01cb60 con 0x7f7a84156460 2023-10-07T07:57:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.153+0000 7f9f98ae7700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f940b4760 0x7f9f9415f830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00bc30 tx=0x7f9f8c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.154+0000 7fdcd6d9d700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdcc4066fb0 0x7fdcc4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.153+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a840ae7c0 con 0x7f7a84156460 2023-10-07T07:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.153+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a841766a0 con 0x7f7a84156460 2023-10-07T07:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.154+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a840716c0 con 0x7f7a84156460 2023-10-07T07:57:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.155+0000 7fdcd6d9d700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdcc4066fb0 0x7fdcc4069460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005c50 tx=0x7fdcd00014b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.155+0000 7ff2b1ffb700 1 -- 172.21.15.7:0/1151251598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2b800c460 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.156+0000 7ff2b1ffb700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff2a0067080 0x7ff2a0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.156+0000 7ff2b1ffb700 1 -- 172.21.15.7:0/1151251598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2b8084dc0 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.156+0000 7f4726e43700 1 -- 172.21.15.7:0/4275061584 shutdown_connections 2023-10-07T07:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.156+0000 7f4726e43700 1 --2- 172.21.15.7:0/4275061584 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f472015e2b0 0x7f472015e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.156+0000 7f4726e43700 1 --2- 172.21.15.7:0/4275061584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47200cb3c0 0x7f47200cb790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.156+0000 7f4726e43700 1 --2- 172.21.15.7:0/4275061584 >> v2:172.21.15.172:3300/0 conn(0x7f47200c64d0 0x7f47200c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.156+0000 7f4726e43700 1 -- 172.21.15.7:0/4275061584 >> 172.21.15.7:0/4275061584 conn(0x7f4720067a80 msgr2=0x7f47200cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.156+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3414140570 con 0x7f34140c64d0 2023-10-07T07:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7ff6397fa700 1 -- 172.21.15.7:0/2420005114 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff63c053cc0 con 0x7ff6440b4270 2023-10-07T07:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.153+0000 7f82113fe700 1 Processor -- start 2023-10-07T07:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.153+0000 7f82113fe700 1 -- start start 2023-10-07T07:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.154+0000 7f82113fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f820c15d410 0x7f820c06df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.154+0000 7f82113fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f820c0681e0 0x7f820c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.154+0000 7f82113fe700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c0b34a0 con 0x7f820c15d7e0 2023-10-07T07:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.154+0000 7f82113fe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c0b40b0 con 0x7f820c0681e0 2023-10-07T07:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.154+0000 7f82113fe700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c0b3780 con 0x7f820c15d410 2023-10-07T07:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.154+0000 7f8210bfd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f820c0681e0 0x7f820c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.155+0000 7f8210bfd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f820c0681e0 0x7f820c0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33824/0 (socket says 172.21.15.7:33824) 2023-10-07T07:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.155+0000 7f8210bfd700 1 -- 172.21.15.7:0/2768154761 learned_addr learned my addr 172.21.15.7:0/2768154761 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.155+0000 7f8210bfd700 1 -- 172.21.15.7:0/2768154761 >> v2:172.21.15.172:3300/0 conn(0x7f820c15d410 msgr2=0x7f820c06df60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.155+0000 7f8210bfd700 1 --2- 172.21.15.7:0/2768154761 >> v2:172.21.15.172:3300/0 conn(0x7f820c15d410 0x7f820c06df60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.155+0000 7f8210bfd700 1 -- 172.21.15.7:0/2768154761 >> v1:172.21.15.7:6789/0 conn(0x7f820c15d7e0 legacy=0x7f820c067ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.155+0000 7f8210bfd700 1 -- 172.21.15.7:0/2768154761 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c0b3a60 con 0x7f820c0681e0 2023-10-07T07:57:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.155+0000 7f8210bfd700 1 --2- 172.21.15.7:0/2768154761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 0x7f820c0b4760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f820c089510 tx=0x7f8204009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6170dc10ecda99 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7f9f7bfff700 1 -- 172.21.15.7:0/282761721 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0092e0 con 0x7f9f940b4760 2023-10-07T07:57:57.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7ff2c1e71700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff2a0067080 0x7ff2a0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7f9f7bfff700 1 -- 172.21.15.7:0/282761721 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c00c7a0 con 0x7f9f940b4760 2023-10-07T07:57:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7f9f7bfff700 1 -- 172.21.15.7:0/282761721 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c00cb60 con 0x7f9f940b4760 2023-10-07T07:57:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7f7a73fff700 1 -- 172.21.15.7:0/1211273246 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a7c01ccc0 con 0x7f7a84156460 2023-10-07T07:57:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7f7a73fff700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7a68066f80 0x7f7a68069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7f7a73fff700 1 -- 172.21.15.7:0/1211273246 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7a7c0860a0 con 0x7f7a84156460 2023-10-07T07:57:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7ff2c1e71700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff2a0067080 0x7ff2a0069530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009aa0 tx=0x7ff2ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.158+0000 7f7a83fff700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7a68066f80 0x7f7a68069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7f34097fa700 1 -- 172.21.15.7:0/464170555 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3404022030 con 0x7f34140c64d0 2023-10-07T07:57:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.158+0000 7f34097fa700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f33f8066f30 0x7f33f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.158+0000 7f34097fa700 1 -- 172.21.15.7:0/464170555 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3404086500 con 0x7f34140c64d0 2023-10-07T07:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.158+0000 7f3418dc8700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f33f8066f30 0x7f33f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.158+0000 7f7a83fff700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7a68066f80 0x7f7a68069430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a78006fd0 tx=0x7f7a78008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9416f280 con 0x7f9f940b4760 2023-10-07T07:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.157+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f94143c10 con 0x7f9f940b4760 2023-10-07T07:57:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.159+0000 7f3418dc8700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f33f8066f30 0x7f33f80693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f34141713b0 tx=0x7f3410007b50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.159+0000 7f4726e43700 1 -- 172.21.15.7:0/4275061584 shutdown_connections 2023-10-07T07:57:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.159+0000 7f4726e43700 1 -- 172.21.15.7:0/4275061584 wait complete. 2023-10-07T07:57:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.158+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9416f0e0 con 0x7f9f940b4760 2023-10-07T07:57:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.161+0000 7f7a73fff700 1 -- 172.21.15.7:0/1211273246 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a7c0532f0 con 0x7f7a84156460 2023-10-07T07:57:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.161+0000 7f4726e43700 1 Processor -- start 2023-10-07T07:57:57.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.161+0000 7f9f7bfff700 1 -- 172.21.15.7:0/282761721 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f8c012070 con 0x7f9f940b4760 2023-10-07T07:57:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.162+0000 7f9f7bfff700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f80066f30 0x7f9f800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.162+0000 7ff2b1ffb700 1 -- 172.21.15.7:0/1151251598 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2b8052090 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.162+0000 7f9f93fff700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f80066f30 0x7f9f800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.163+0000 7f9f93fff700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f80066f30 0x7f9f800693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9f840098e0 tx=0x7f9f84016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.165+0000 7f9f7bfff700 1 -- 172.21.15.7:0/282761721 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f8c055880 con 0x7f9f940b4760 2023-10-07T07:57:57.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.161+0000 7f4726e43700 1 -- start start 2023-10-07T07:57:57.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.162+0000 7f4726e43700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200c64d0 0x7f47200ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.162+0000 7f4726e43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47200cb3c0 0x7f472013fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.162+0000 7f4726e43700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f472015e2b0 0x7f472013d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.162+0000 7f4726e43700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47200ad430 con 0x7f47200cb3c0 2023-10-07T07:57:57.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.162+0000 7f4726e43700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4720140530 con 0x7f472015e2b0 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.162+0000 7f4726e43700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472013d880 con 0x7f47200c64d0 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.166+0000 7f47253e0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f472015e2b0 0x7f472013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.166+0000 7f47253e0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f472015e2b0 0x7f472013d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39502/0 (socket says 172.21.15.7:39502) 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.166+0000 7f47253e0700 1 -- 172.21.15.7:0/3300868194 learned_addr learned my addr 172.21.15.7:0/3300868194 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.166+0000 7f4724bdf700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200c64d0 0x7f47200ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.166+0000 7f47253e0700 1 -- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200c64d0 msgr2=0x7f47200ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.166+0000 7f47253e0700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200c64d0 0x7f47200ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.166+0000 7f47253e0700 1 -- 172.21.15.7:0/3300868194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47200cb3c0 msgr2=0x7f472013fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.166+0000 7f47253e0700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47200cb3c0 0x7f472013fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.166+0000 7f47253e0700 1 -- 172.21.15.7:0/3300868194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c0094f0 con 0x7f472015e2b0 2023-10-07T07:57:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.167+0000 7f47253e0700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f472015e2b0 0x7f472013d340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f471800cdb0 tx=0x7f47180077d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.169+0000 7f34097fa700 1 -- 172.21.15.7:0/464170555 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34040536d0 con 0x7f34140c64d0 2023-10-07T07:57:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.169+0000 7f4714ff9700 1 -- 172.21.15.7:0/3300868194 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471800aa80 con 0x7f472015e2b0 2023-10-07T07:57:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.169+0000 7f4714ff9700 1 -- 172.21.15.7:0/3300868194 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471800abe0 con 0x7f472015e2b0 2023-10-07T07:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.169+0000 7f4714ff9700 1 -- 172.21.15.7:0/3300868194 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718017430 con 0x7f472015e2b0 2023-10-07T07:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.169+0000 7f9f7bfff700 1 -- 172.21.15.7:0/282761721 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f8c0514c0 con 0x7f9f940b4760 2023-10-07T07:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.170+0000 7f820affd700 1 -- 172.21.15.7:0/2768154761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820400e070 con 0x7f820c0681e0 2023-10-07T07:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.170+0000 7f820affd700 1 -- 172.21.15.7:0/2768154761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820400ba40 con 0x7f820c0681e0 2023-10-07T07:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.170+0000 7f820affd700 1 -- 172.21.15.7:0/2768154761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040092e0 con 0x7f820c0681e0 2023-10-07T07:57:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.170+0000 7fdcbeffd700 1 -- 172.21.15.7:0/2936500964 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcc8052ec0 con 0x7fdcd81570a0 2023-10-07T07:57:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.171+0000 7f82113fe700 1 -- 172.21.15.7:0/2768154761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 msgr2=0x7f820c0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.171+0000 7f82113fe700 1 --2- 172.21.15.7:0/2768154761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 0x7f820c0b4760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f820c089510 tx=0x7f8204009840 comp rx=0 tx=0).stop 2023-10-07T07:57:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.171+0000 7f82113fe700 1 -- 172.21.15.7:0/2768154761 shutdown_connections 2023-10-07T07:57:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.171+0000 7f82113fe700 1 --2- 172.21.15.7:0/2768154761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 0x7f820c0b4760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.171+0000 7f82113fe700 1 --2- 172.21.15.7:0/2768154761 >> v2:172.21.15.172:3300/0 conn(0x7f820c15d410 0x7f820c06df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.171+0000 7f82113fe700 1 -- 172.21.15.7:0/2768154761 >> 172.21.15.7:0/2768154761 conn(0x7f820c0d0120 msgr2=0x7f820c0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.177+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47200ac820 con 0x7f472015e2b0 2023-10-07T07:57:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.177+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47200acde0 con 0x7f472015e2b0 2023-10-07T07:57:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.172+0000 7f82113fe700 1 -- 172.21.15.7:0/2768154761 shutdown_connections 2023-10-07T07:57:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.172+0000 7f82113fe700 1 -- 172.21.15.7:0/2768154761 wait complete. 2023-10-07T07:57:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.174+0000 7f82113fe700 1 Processor -- start 2023-10-07T07:57:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.174+0000 7f82113fe700 1 -- start start 2023-10-07T07:57:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.175+0000 7f82113fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 0x7f820c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.175+0000 7f82113fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f820c15d410 0x7f820c0876c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.175+0000 7f82113fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f820c15d7e0 0x7f820c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.175+0000 7f82113fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c06cfe0 con 0x7f820c0681e0 2023-10-07T07:57:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.175+0000 7f82113fe700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c06c3d0 con 0x7f820c15d410 2023-10-07T07:57:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.175+0000 7f82113fe700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c06c6b0 con 0x7f820c15d7e0 2023-10-07T07:57:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.176+0000 7f820bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 0x7f820c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.176+0000 7f820bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 0x7f820c06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33852/0 (socket says 172.21.15.7:33852) 2023-10-07T07:57:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.176+0000 7f820bfff700 1 -- 172.21.15.7:0/2767187793 learned_addr learned my addr 172.21.15.7:0/2767187793 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.176+0000 7f820b7fe700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f820c15d410 0x7f820c0876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.176+0000 7f820bfff700 1 -- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f820c15d7e0 msgr2=0x7f820c0cb620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T07:57:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.176+0000 7f820bfff700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f820c15d7e0 0x7f820c0cb620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.176+0000 7f820bfff700 1 -- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f820c15d410 msgr2=0x7f820c0876c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.176+0000 7f820bfff700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f820c15d410 0x7f820c0876c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.176+0000 7f820bfff700 1 -- 172.21.15.7:0/2767187793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204009ae0 con 0x7f820c0681e0 2023-10-07T07:57:57.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.177+0000 7f820bfff700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 0x7f820c06bdf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00b5e0 tx=0x7f81fc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.180+0000 7f4714ff9700 1 -- 172.21.15.7:0/3300868194 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4718003020 con 0x7f472015e2b0 2023-10-07T07:57:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.181+0000 7f4714ff9700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f470c067080 0x7f470c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.181+0000 7f4714ff9700 1 -- 172.21.15.7:0/3300868194 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4718013070 con 0x7f472015e2b0 2023-10-07T07:57:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.178+0000 7f81f3fff700 1 -- 172.21.15.7:0/2767187793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00f510 con 0x7f820c0681e0 2023-10-07T07:57:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.178+0000 7f81f3fff700 1 -- 172.21.15.7:0/2767187793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc00f690 con 0x7f820c0681e0 2023-10-07T07:57:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.178+0000 7f81f3fff700 1 -- 172.21.15.7:0/2767187793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00fa50 con 0x7f820c0681e0 2023-10-07T07:57:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.181+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c06c990 con 0x7f820c0681e0 2023-10-07T07:57:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.181+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c14da10 con 0x7f820c0681e0 2023-10-07T07:57:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.185+0000 7f81f3fff700 1 -- 172.21.15.7:0/2767187793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f81fc01a400 con 0x7f820c0681e0 2023-10-07T07:57:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.186+0000 7f81f3fff700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f81f8067080 0x7f81f8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:57.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.186+0000 7f81f3fff700 1 -- 172.21.15.7:0/2767187793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f81fc084080 con 0x7f820c0681e0 2023-10-07T07:57:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.186+0000 7f820b7fe700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f81f8067080 0x7f81f8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.187+0000 7f820b7fe700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f81f8067080 0x7f81f8069530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f820c14d9a0 tx=0x7f8200009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.193+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c04c3c0 con 0x7f820c0681e0 2023-10-07T07:57:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.193+0000 7f4724bdf700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f470c067080 0x7f470c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.194+0000 7f4724bdf700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f470c067080 0x7f470c069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47201612e0 tx=0x7f471c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.195+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47200ad5a0 con 0x7f472015e2b0 2023-10-07T07:57:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.201+0000 7f4714ff9700 1 -- 172.21.15.7:0/3300868194 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47180518e0 con 0x7f472015e2b0 2023-10-07T07:57:57.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.203+0000 7f81f3fff700 1 -- 172.21.15.7:0/2767187793 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81fc051350 con 0x7f820c0681e0 2023-10-07T07:57:57.566 INFO:teuthology.orchestra.run.smithi007.stdout:55834574851 2023-10-07T07:57:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.561+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff2bc0716c0 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.562+0000 7ff2b1ffb700 1 -- 172.21.15.7:0/1151251598 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff2b80556b0 con 0x7ff2bc0cb3c0 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff2a0067080 msgr2=0x7ff2a0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff2a0067080 0x7ff2a0069530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009aa0 tx=0x7ff2ac008040 comp rx=0 tx=0).stop 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc0cb3c0 msgr2=0x7ff2bc1573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc0cb3c0 0x7ff2bc1573e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8006d40 tx=0x7ff2b8009250 comp rx=0 tx=0).stop 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 shutdown_connections 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff2a0067080 0x7ff2a0069530 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2bc15e2b0 0x7ff2bc153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2bc0c64d0 0x7ff2bc156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 --2- 172.21.15.7:0/1151251598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2bc0cb3c0 0x7ff2bc1573e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.565+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 >> 172.21.15.7:0/1151251598 conn(0x7ff2bc067a80 msgr2=0x7ff2bc0c0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.566+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 shutdown_connections 2023-10-07T07:57:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.566+0000 7ff2c40d5700 1 -- 172.21.15.7:0/1151251598 wait complete. 2023-10-07T07:57:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.581+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff6440716c0 con 0x7ff6440b4270 2023-10-07T07:57:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.581+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7a840cb970 con 0x7f7a84156460 2023-10-07T07:57:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.584+0000 7ff6397fa700 1 -- 172.21.15.7:0/2420005114 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff63c0572e0 con 0x7ff6440b4270 2023-10-07T07:57:57.586 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-10-07T07:57:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.584+0000 7f7a73fff700 1 -- 172.21.15.7:0/1211273246 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a7c056910 con 0x7f7a84156460 2023-10-07T07:57:57.587 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-10-07T07:57:57.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2023-10-07T07:57:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff628067080 msgr2=0x7ff628069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff628067080 0x7ff628069530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff644131ac0 tx=0x7ff640009040 comp rx=0 tx=0).stop 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6440b4270 msgr2=0x7ff644158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6440b4270 0x7ff644158280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff63c016000 tx=0x7ff63c019000 comp rx=0 tx=0).stop 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 shutdown_connections 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff628067080 0x7ff628069530 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff644159380 0x7ff644179ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6440b4270 0x7ff644158280 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 --2- 172.21.15.7:0/2420005114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6440d0250 0x7ff644157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 >> 172.21.15.7:0/2420005114 conn(0x7ff644067be0 msgr2=0x7ff6440b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 shutdown_connections 2023-10-07T07:57:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7a68066f80 msgr2=0x7f7a68069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7a68066f80 0x7f7a68069430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a78006fd0 tx=0x7f7a78008040 comp rx=0 tx=0).stop 2023-10-07T07:57:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a84156460 msgr2=0x7f7a840b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a84156460 0x7f7a840b0150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a8406f910 tx=0x7f7a7c019000 comp rx=0 tx=0).stop 2023-10-07T07:57:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 shutdown_connections 2023-10-07T07:57:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7a68066f80 0x7f7a68069430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a84156460 0x7f7a840b0150 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a840c6be0 0x7f7a840b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 --2- 172.21.15.7:0/1211273246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a8413b920 0x7f7a840ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 >> 172.21.15.7:0/1211273246 conn(0x7f7a84067a80 msgr2=0x7f7a84158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.588+0000 7ff64b901700 1 -- 172.21.15.7:0/2420005114 wait complete. 2023-10-07T07:57:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.589+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 shutdown_connections 2023-10-07T07:57:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.589+0000 7f7a8a7e7700 1 -- 172.21.15.7:0/1211273246 wait complete. 2023-10-07T07:57:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.597+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdcb8005560 con 0x7fdcd81570a0 2023-10-07T07:57:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.597+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9f9416f0e0 con 0x7f9f940b4760 2023-10-07T07:57:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.598+0000 7fdcbeffd700 1 -- 172.21.15.7:0/2936500964 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdcc8029030 con 0x7fdcd81570a0 2023-10-07T07:57:57.599 INFO:teuthology.orchestra.run.smithi007.stdout:55834574852 2023-10-07T07:57:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.597+0000 7f9f7bfff700 1 -- 172.21.15.7:0/282761721 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f9416f0e0 con 0x7f9f940b4760 2023-10-07T07:57:57.600 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-10-07T07:57:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.599+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3414140570 con 0x7f34140c64d0 2023-10-07T07:57:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.601+0000 7f34097fa700 1 -- 172.21.15.7:0/464170555 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3404056cf0 con 0x7f34140c64d0 2023-10-07T07:57:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.601+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f80066f30 msgr2=0x7f9f800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.601+0000 7f9f9a54a700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f80066f30 0x7f9f800693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9f840098e0 tx=0x7f9f84016000 comp rx=0 tx=0).stop 2023-10-07T07:57:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.601+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f940b4760 msgr2=0x7f9f9415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.601+0000 7f9f9a54a700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f940b4760 0x7f9f9415f830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00bc30 tx=0x7f9f8c009840 comp rx=0 tx=0).stop 2023-10-07T07:57:57.602 INFO:teuthology.orchestra.run.smithi007.stdout:55834574851 2023-10-07T07:57:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.601+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdcc4066fb0 msgr2=0x7fdcc4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.601+0000 7fdcdd12b700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdcc4066fb0 0x7fdcc4069460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005c50 tx=0x7fdcd00014b0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.601+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd81570a0 msgr2=0x7fdcd80b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.601+0000 7fdcdd12b700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd81570a0 0x7fdcd80b1800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8015000 tx=0x7fdcc8016000 comp rx=0 tx=0).stop 2023-10-07T07:57:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.602+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 shutdown_connections 2023-10-07T07:57:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.602+0000 7f9f9a54a700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f80066f30 0x7f9f800693e0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9f840098e0 tx=0x7f9f84016000 comp rx=0 tx=0).stop 2023-10-07T07:57:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.602+0000 7f9f9a54a700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f940b4760 0x7f9f9415f830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.602+0000 7f9f9a54a700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f9406eeb0 0x7f9f941436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.602+0000 7f9f9a54a700 1 --2- 172.21.15.7:0/282761721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f9406f280 0x7f9f9415f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.602+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 >> 172.21.15.7:0/282761721 conn(0x7f9f94067be0 msgr2=0x7f9f940cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 shutdown_connections 2023-10-07T07:57:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7fdcdd12b700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdcc4066fb0 0x7fdcc4069460 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005c50 tx=0x7fdcd00014b0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7fdcdd12b700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcd8157470 0x7fdcd813cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7fdcdd12b700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcd806bb60 0x7fdcd8139930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7fdcdd12b700 1 --2- 172.21.15.7:0/2936500964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd81570a0 0x7fdcd80b1800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 >> 172.21.15.7:0/2936500964 conn(0x7fdcd8067ac0 msgr2=0x7fdcd80687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f33f8066f30 msgr2=0x7f33f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7f341b82d700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f33f8066f30 0x7f33f80693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f34141713b0 tx=0x7f3410007b50 comp rx=0 tx=0).stop 2023-10-07T07:57:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34140c64d0 msgr2=0x7f34140b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.603+0000 7f341b82d700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34140c64d0 0x7f34140b1550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f341406cc40 tx=0x7f3404016000 comp rx=0 tx=0).stop 2023-10-07T07:57:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 shutdown_connections 2023-10-07T07:57:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7f341b82d700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f33f8066f30 0x7f33f80693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7f341b82d700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f341415e9c0 0x7f34140af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7f341b82d700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34140c64d0 0x7f34140b1550 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f341406cc40 tx=0x7f3404016000 comp rx=0 tx=0).stop 2023-10-07T07:57:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7f341b82d700 1 --2- 172.21.15.7:0/464170555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34140cb3c0 0x7f34140aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 >> 172.21.15.7:0/464170555 conn(0x7f3414067a80 msgr2=0x7f341415ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 shutdown_connections 2023-10-07T07:57:57.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 shutdown_connections 2023-10-07T07:57:57.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7f341b82d700 1 -- 172.21.15.7:0/464170555 wait complete. 2023-10-07T07:57:57.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.604+0000 7fdcdd12b700 1 -- 172.21.15.7:0/2936500964 wait complete. 2023-10-07T07:57:57.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.606+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 shutdown_connections 2023-10-07T07:57:57.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-10-07T07:57:57.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.609+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f47200716c0 con 0x7f472015e2b0 2023-10-07T07:57:57.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.609+0000 7f9f9a54a700 1 -- 172.21.15.7:0/282761721 wait complete. 2023-10-07T07:57:57.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.609+0000 7f4714ff9700 1 -- 172.21.15.7:0/3300868194 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47180078f0 con 0x7f472015e2b0 2023-10-07T07:57:57.611 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-10-07T07:57:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f470c067080 msgr2=0x7f470c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f470c067080 0x7f470c069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47201612e0 tx=0x7f471c017000 comp rx=0 tx=0).stop 2023-10-07T07:57:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f472015e2b0 msgr2=0x7f472013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f472015e2b0 0x7f472013d340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f471800cdb0 tx=0x7f47180077d0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 shutdown_connections 2023-10-07T07:57:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f470c067080 0x7f470c069530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f472015e2b0 0x7f472013d340 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200c64d0 0x7f47200ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 --2- 172.21.15.7:0/3300868194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47200cb3c0 0x7f472013fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.612+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 >> 172.21.15.7:0/3300868194 conn(0x7f4720067a80 msgr2=0x7f47200c8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-07T07:57:57.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.613+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 shutdown_connections 2023-10-07T07:57:57.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.613+0000 7f4726e43700 1 -- 172.21.15.7:0/3300868194 wait complete. 2023-10-07T07:57:57.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.615+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f820c0b3d40 con 0x7f820c0681e0 2023-10-07T07:57:57.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.616+0000 7f81f3fff700 1 -- 172.21.15.7:0/2767187793 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f81fc054970 con 0x7f820c0681e0 2023-10-07T07:57:57.616 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-10-07T07:57:57.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.619+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f81f8067080 msgr2=0x7f81f8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.619+0000 7f82113fe700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f81f8067080 0x7f81f8069530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f820c14d9a0 tx=0x7f8200009040 comp rx=0 tx=0).stop 2023-10-07T07:57:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.619+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 msgr2=0x7f820c06bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.619+0000 7f82113fe700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 0x7f820c06bdf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00b5e0 tx=0x7f81fc0090f0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.620+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 shutdown_connections 2023-10-07T07:57:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.620+0000 7f82113fe700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f81f8067080 0x7f81f8069530 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.620+0000 7f82113fe700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f820c15d7e0 0x7f820c0cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.620+0000 7f82113fe700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f820c0681e0 0x7f820c06bdf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.620+0000 7f82113fe700 1 --2- 172.21.15.7:0/2767187793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f820c15d410 0x7f820c0876c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.620+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 >> 172.21.15.7:0/2767187793 conn(0x7f820c0d0120 msgr2=0x7f820c06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:57.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.620+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 shutdown_connections 2023-10-07T07:57:57.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:57.620+0000 7f82113fe700 1 -- 172.21.15.7:0/2767187793 wait complete. 2023-10-07T07:57:57.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2023-10-07T07:57:57.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2023-10-07T07:57:57.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-10-07T07:57:57.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-10-07T07:57:57.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-10-07T07:57:57.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:57:57.684 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:57:58.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-07T07:57:58.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-07T07:57:58.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-07T07:57:58.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-07T07:57:58.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-07T07:57:58.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-07T07:57:58.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-07T07:57:58.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-07T07:57:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.808+0000 7f769d01c700 1 Processor -- start 2023-10-07T07:57:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.808+0000 7f769d01c700 1 -- start start 2023-10-07T07:57:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769d01c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7698165e10 0x7f76980c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769d01c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76981661e0 0x7f76980878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769d01c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f769806dac0 0x7f769806df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769d01c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769806e470 con 0x7f76981661e0 2023-10-07T07:57:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769d01c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76980c9b90 con 0x7f769806dac0 2023-10-07T07:57:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769d01c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7698087de0 con 0x7f7698165e10 2023-10-07T07:57:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76981661e0 0x7f76980878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76981661e0 0x7f76980878a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33860/0 (socket says 172.21.15.7:33860) 2023-10-07T07:57:58.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769659c700 1 -- 172.21.15.7:0/3485255388 learned_addr learned my addr 172.21.15.7:0/3485255388 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.811+0000 7f7696d9d700 1 --2- 172.21.15.7:0/3485255388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7698165e10 0x7f76980c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.810+0000 7f769759e700 1 --2- 172.21.15.7:0/3485255388 >> v2:172.21.15.172:3300/0 conn(0x7f769806dac0 0x7f769806df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.811+0000 7f7696d9d700 1 -- 172.21.15.7:0/3485255388 >> v2:172.21.15.172:3300/0 conn(0x7f769806dac0 msgr2=0x7f769806df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.811+0000 7f7696d9d700 1 --2- 172.21.15.7:0/3485255388 >> v2:172.21.15.172:3300/0 conn(0x7f769806dac0 0x7f769806df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.811+0000 7f7696d9d700 1 -- 172.21.15.7:0/3485255388 >> v2:172.21.15.7:3300/0 conn(0x7f76981661e0 msgr2=0x7f76980878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.811+0000 7f7696d9d700 1 --2- 172.21.15.7:0/3485255388 >> v2:172.21.15.7:3300/0 conn(0x7f76981661e0 0x7f76980878a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.811+0000 7f7696d9d700 1 -- 172.21.15.7:0/3485255388 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76980c7b80 con 0x7f7698165e10 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.811+0000 7f769659c700 1 --2- 172.21.15.7:0/3485255388 >> v2:172.21.15.7:3300/0 conn(0x7f76981661e0 0x7f76980878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.811+0000 7f769759e700 1 --2- 172.21.15.7:0/3485255388 >> v2:172.21.15.172:3300/0 conn(0x7f769806dac0 0x7f769806df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.812+0000 7f7696d9d700 1 --2- 172.21.15.7:0/3485255388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7698165e10 0x7f76980c7490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f769806da10 tx=0x7f769000d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a69c77483a50464 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.812+0000 7f7695d9b700 1 -- 172.21.15.7:0/3485255388 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769000b300 con 0x7f7698165e10 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.812+0000 7f7695d9b700 1 -- 172.21.15.7:0/3485255388 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769000ca10 con 0x7f7698165e10 2023-10-07T07:57:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.812+0000 7f7695d9b700 1 -- 172.21.15.7:0/3485255388 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769000cdd0 con 0x7f7698165e10 2023-10-07T07:57:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.813+0000 7f769d01c700 1 -- 172.21.15.7:0/3485255388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7698165e10 msgr2=0x7f76980c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.813+0000 7f769d01c700 1 --2- 172.21.15.7:0/3485255388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7698165e10 0x7f76980c7490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f769806da10 tx=0x7f769000d040 comp rx=0 tx=0).stop 2023-10-07T07:57:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.813+0000 7f769d01c700 1 -- 172.21.15.7:0/3485255388 shutdown_connections 2023-10-07T07:57:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.813+0000 7f769d01c700 1 --2- 172.21.15.7:0/3485255388 >> v2:172.21.15.172:3300/0 conn(0x7f769806dac0 0x7f769806df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.813+0000 7f769d01c700 1 --2- 172.21.15.7:0/3485255388 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7698165e10 0x7f76980c7490 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f769806da10 tx=0x7f769000d040 comp rx=0 tx=0).stop 2023-10-07T07:57:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.814+0000 7f769d01c700 1 --2- 172.21.15.7:0/3485255388 >> v2:172.21.15.7:3300/0 conn(0x7f76981661e0 0x7f76980878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.814+0000 7f769d01c700 1 -- 172.21.15.7:0/3485255388 >> 172.21.15.7:0/3485255388 conn(0x7f76980cbe90 msgr2=0x7f76980cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.814+0000 7f769d01c700 1 -- 172.21.15.7:0/3485255388 shutdown_connections 2023-10-07T07:57:58.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.814+0000 7f769d01c700 1 -- 172.21.15.7:0/3485255388 wait complete. 2023-10-07T07:57:58.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.816+0000 7f769d01c700 1 Processor -- start 2023-10-07T07:57:58.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.816+0000 7f769d01c700 1 -- start start 2023-10-07T07:57:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.818+0000 7f769d01c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769806dac0 0x7f7698160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.818+0000 7f769d01c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76981661e0 0x7f769817a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.818+0000 7f769d01c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76980c1ee0 0x7f76980c2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.818+0000 7f769d01c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7698160fd0 con 0x7f76981661e0 2023-10-07T07:57:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.818+0000 7f769d01c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76980c28a0 con 0x7f76980c1ee0 2023-10-07T07:57:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.818+0000 7f769d01c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76980b0530 con 0x7f769806dac0 2023-10-07T07:57:58.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.818+0000 7f7696d9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769806dac0 0x7f7698160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.818+0000 7f7696d9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769806dac0 0x7f7698160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:60970/0 (socket says 172.21.15.7:60970) 2023-10-07T07:57:58.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.818+0000 7f7696d9d700 1 -- 172.21.15.7:0/2346079269 learned_addr learned my addr 172.21.15.7:0/2346079269 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.819+0000 7f769759e700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76980c1ee0 0x7f76980c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.819+0000 7f7696d9d700 1 -- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76980c1ee0 msgr2=0x7f76980c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.819+0000 7f7696d9d700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76980c1ee0 0x7f76980c2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.819+0000 7f7696d9d700 1 -- 172.21.15.7:0/2346079269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76981661e0 msgr2=0x7f769817a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.819+0000 7f7696d9d700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76981661e0 0x7f769817a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.819+0000 7f7696d9d700 1 -- 172.21.15.7:0/2346079269 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7688008570 con 0x7f769806dac0 2023-10-07T07:57:58.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.820+0000 7f7696d9d700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769806dac0 0x7f7698160a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7698087f20 tx=0x7f769001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.821+0000 7f767effd700 1 -- 172.21.15.7:0/2346079269 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690009760 con 0x7f769806dac0 2023-10-07T07:57:58.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.821+0000 7f767effd700 1 -- 172.21.15.7:0/2346079269 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7690023070 con 0x7f769806dac0 2023-10-07T07:57:58.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.821+0000 7f767effd700 1 -- 172.21.15.7:0/2346079269 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690009b10 con 0x7f769806dac0 2023-10-07T07:57:58.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.821+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769000ba40 con 0x7f769806dac0 2023-10-07T07:57:58.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.821+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76980af920 con 0x7f769806dac0 2023-10-07T07:57:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.823+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76780052b0 con 0x7f769806dac0 2023-10-07T07:57:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.824+0000 7f767effd700 1 -- 172.21.15.7:0/2346079269 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f769001b070 con 0x7f769806dac0 2023-10-07T07:57:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.824+0000 7f767effd700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7684067080 0x7f7684069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.824+0000 7f767effd700 1 -- 172.21.15.7:0/2346079269 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f769008de70 con 0x7f769806dac0 2023-10-07T07:57:58.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.825+0000 7f769659c700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7684067080 0x7f7684069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.826+0000 7f769659c700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7684067080 0x7f7684069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7698087aa0 tx=0x7f768800c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.829+0000 7f767effd700 1 -- 172.21.15.7:0/2346079269 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f769005b0c0 con 0x7f769806dac0 2023-10-07T07:57:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.851+0000 7fb02dcea700 1 Processor -- start 2023-10-07T07:57:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.852+0000 7fb02dcea700 1 -- start start 2023-10-07T07:57:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.853+0000 7fb02dcea700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0281563a0 0x7fb02806dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.853+0000 7fb02dcea700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0280c86b0 con 0x7fb0280c7630 2023-10-07T07:57:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.853+0000 7fb02dcea700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0280c7b30 con 0x7fb028156770 2023-10-07T07:57:58.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.853+0000 7fb02dcea700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02814c950 con 0x7fb0281563a0 2023-10-07T07:57:58.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.853+0000 7fb026ffd700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb028156770 0x7fb0280c6f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:60042/0 (socket says 172.21.15.7:60042) 2023-10-07T07:57:58.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.853+0000 7fb026ffd700 1 -- 172.21.15.7:0/806721916 learned_addr learned my addr 172.21.15.7:0/806721916 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:58.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.853+0000 7fb0277fe700 1 --2- 172.21.15.7:0/806721916 >> v2:172.21.15.7:3300/0 conn(0x7fb0281563a0 0x7fb02806dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.854+0000 7fb0267fc700 1 -- 172.21.15.7:0/806721916 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113934791 0 0) 0x7fb0280c86b0 con 0x7fb0280c7630 2023-10-07T07:57:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.854+0000 7fb0277fe700 1 -- 172.21.15.7:0/806721916 >> v1:172.21.15.172:6789/0 conn(0x7fb028156770 legacy=0x7fb0280c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.854+0000 7fb0277fe700 1 -- 172.21.15.7:0/806721916 >> v1:172.21.15.7:6789/0 conn(0x7fb0280c7630 legacy=0x7fb0280c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.854+0000 7fb0277fe700 1 -- 172.21.15.7:0/806721916 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0280c7e10 con 0x7fb0281563a0 2023-10-07T07:57:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.854+0000 7fb0277fe700 1 --2- 172.21.15.7:0/806721916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0281563a0 0x7fb02806dd70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb01800aa90 tx=0x7fb01800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ce1d5ce69d8980 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.854+0000 7fb0267fc700 1 -- 172.21.15.7:0/806721916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01800ca40 con 0x7fb0281563a0 2023-10-07T07:57:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.854+0000 7fb0267fc700 1 -- 172.21.15.7:0/806721916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01800cba0 con 0x7fb0281563a0 2023-10-07T07:57:58.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.854+0000 7fb0267fc700 1 -- 172.21.15.7:0/806721916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018018400 con 0x7fb0281563a0 2023-10-07T07:57:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.855+0000 7f03713ac700 1 Processor -- start 2023-10-07T07:57:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.855+0000 7fb02dcea700 1 -- 172.21.15.7:0/806721916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0281563a0 msgr2=0x7fb02806dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.855+0000 7fb02dcea700 1 --2- 172.21.15.7:0/806721916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0281563a0 0x7fb02806dd70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb01800aa90 tx=0x7fb01800a330 comp rx=0 tx=0).stop 2023-10-07T07:57:58.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.855+0000 7f03713ac700 1 -- start start 2023-10-07T07:57:58.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.856+0000 7f03713ac700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f036c0c89e0 0x7f036c168600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.856+0000 7f03713ac700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c06ae70 con 0x7f036c146f60 2023-10-07T07:57:58.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.856+0000 7f03713ac700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c06b150 con 0x7f036c147330 2023-10-07T07:57:58.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.856+0000 7f03713ac700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c06b430 con 0x7f036c0c89e0 2023-10-07T07:57:58.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.856+0000 7f036affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f036c146f60 0x7f036c06a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36046/0 (socket says 172.21.15.7:36046) 2023-10-07T07:57:58.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.856+0000 7f036affd700 1 -- 172.21.15.7:0/1818572879 learned_addr learned my addr 172.21.15.7:0/1818572879 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f036b7fe700 1 --2- 172.21.15.7:0/1818572879 >> v2:172.21.15.7:3300/0 conn(0x7f036c0c89e0 0x7f036c168600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202887485 0 0) 0x7f036c06ae70 con 0x7f036c146f60 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0354003570 con 0x7f036c146f60 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993456125 0 0) 0x7f0354003570 con 0x7f036c146f60 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 >> v2:172.21.15.7:3300/0 conn(0x7f036c0c89e0 msgr2=0x7f036c168600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 --2- 172.21.15.7:0/1818572879 >> v2:172.21.15.7:3300/0 conn(0x7f036c0c89e0 0x7f036c168600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 >> v1:172.21.15.172:6789/0 conn(0x7f036c147330 legacy=0x7f036c0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c06b710 con 0x7f036c146f60 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f0360001470 con 0x7f036c146f60 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 >> v1:172.21.15.7:6789/0 conn(0x7f036c146f60 legacy=0x7f036c06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 --2- 172.21.15.7:0/1818572879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0354003ae0 0x7f0354003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 --2- 172.21.15.7:0/1818572879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0354004810 0x7f0354004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 --2- 172.21.15.7:0/1818572879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f035400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c06b430 con 0x7f03540052e0 2023-10-07T07:57:58.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035400c1e0 con 0x7f0354004810 2023-10-07T07:57:58.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.857+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035400c3a0 con 0x7f0354003ae0 2023-10-07T07:57:58.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7f036b7fe700 1 --2- 172.21.15.7:0/1818572879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f035400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7fb02dcea700 1 -- 172.21.15.7:0/806721916 shutdown_connections 2023-10-07T07:57:58.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7fb02dcea700 1 --2- 172.21.15.7:0/806721916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0281563a0 0x7fb02806dd70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7fb02dcea700 1 -- 172.21.15.7:0/806721916 >> 172.21.15.7:0/806721916 conn(0x7fb0280cbe90 msgr2=0x7fb0280cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:58.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7f036b7fe700 1 -- 172.21.15.7:0/1818572879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0354003ae0 msgr2=0x7f0354003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:58.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7f036b7fe700 1 --2- 172.21.15.7:0/1818572879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0354003ae0 0x7f0354003ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7f036b7fe700 1 -- 172.21.15.7:0/1818572879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0354004810 msgr2=0x7f0354004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T07:57:58.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7f036b7fe700 1 --2- 172.21.15.7:0/1818572879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0354004810 0x7f0354004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7f036b7fe700 1 -- 172.21.15.7:0/1818572879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c06b710 con 0x7f03540052e0 2023-10-07T07:57:58.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7f036b7fe700 1 -- 172.21.15.7:0/1818572879 reap_dead start 2023-10-07T07:57:58.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.858+0000 7f036b7fe700 1 --2- 172.21.15.7:0/1818572879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f035400ba60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0364007440 tx=0x7f0364009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb61ff855021778 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.859+0000 7fc6dab47700 1 Processor -- start 2023-10-07T07:57:58.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.859+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364009a00 con 0x7f03540052e0 2023-10-07T07:57:58.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.859+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364007c90 con 0x7f03540052e0 2023-10-07T07:57:58.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.859+0000 7f0369ffb700 1 -- 172.21.15.7:0/1818572879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364014690 con 0x7f03540052e0 2023-10-07T07:57:58.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.859+0000 7fb02dcea700 1 -- 172.21.15.7:0/806721916 shutdown_connections 2023-10-07T07:57:58.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.859+0000 7ff88d4eb700 1 Processor -- start 2023-10-07T07:57:58.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.859+0000 7ff88d4eb700 1 -- start start 2023-10-07T07:57:58.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.860+0000 7f03713ac700 1 -- 172.21.15.7:0/1818572879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 msgr2=0x7f035400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.860+0000 7f03713ac700 1 --2- 172.21.15.7:0/1818572879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f035400ba60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0364007440 tx=0x7f0364009710 comp rx=0 tx=0).stop 2023-10-07T07:57:58.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.859+0000 7fc6dab47700 1 -- start start 2023-10-07T07:57:58.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.860+0000 7f03713ac700 1 -- 172.21.15.7:0/1818572879 shutdown_connections 2023-10-07T07:57:58.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.860+0000 7f03713ac700 1 --2- 172.21.15.7:0/1818572879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f035400ba60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.860+0000 7f03713ac700 1 -- 172.21.15.7:0/1818572879 >> 172.21.15.7:0/1818572879 conn(0x7f036c068bd0 msgr2=0x7f036c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.860+0000 7fb02dcea700 1 -- 172.21.15.7:0/806721916 wait complete. 2023-10-07T07:57:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7ff88d4eb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff8880c9af0 0x7ff88815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7ff88d4eb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88815a330 con 0x7ff888063230 2023-10-07T07:57:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7ff88d4eb700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88815a610 con 0x7ff8880c9ec0 2023-10-07T07:57:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7ff88d4eb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8880b03a0 con 0x7ff8880c9af0 2023-10-07T07:57:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7fc6dab47700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6d406bd30 0x7fc6d4168880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7fc6dab47700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d415eed0 con 0x7fc6d415dd80 2023-10-07T07:57:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7fc6dab47700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d415e2c0 con 0x7fc6d4161830 2023-10-07T07:57:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7fc6dab47700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d415e5a0 con 0x7fc6d406bd30 2023-10-07T07:57:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7fc6d90e4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6d406bd30 0x7fc6d4168880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7fc6d3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc6d415dd80 0x7fc6d406b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36056/0 (socket says 172.21.15.7:36056) 2023-10-07T07:57:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7ff8877fe700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff8880c9ec0 0x7ff888158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:60050/0 (socket says 172.21.15.7:60050) 2023-10-07T07:57:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7ff8877fe700 1 -- 172.21.15.7:0/3234461166 learned_addr learned my addr 172.21.15.7:0/3234461166 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.861+0000 7fc6d3fff700 1 -- 172.21.15.7:0/2320174236 learned_addr learned my addr 172.21.15.7:0/2320174236 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7fb02dcea700 1 Processor -- start 2023-10-07T07:57:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7fc6d37fe700 1 -- 172.21.15.7:0/2320174236 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692093930 0 0) 0x7fc6d415eed0 con 0x7fc6d415dd80 2023-10-07T07:57:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7fc6d37fe700 1 -- 172.21.15.7:0/2320174236 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6bc003570 con 0x7fc6d415dd80 2023-10-07T07:57:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7fc6d90e4700 1 -- 172.21.15.7:0/2320174236 >> v1:172.21.15.172:6789/0 conn(0x7fc6d4161830 legacy=0x7fc6d4161c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7fc6d90e4700 1 -- 172.21.15.7:0/2320174236 >> v1:172.21.15.7:6789/0 conn(0x7fc6d415dd80 legacy=0x7fc6d406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7fc6d90e4700 1 -- 172.21.15.7:0/2320174236 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d415e880 con 0x7fc6d406bd30 2023-10-07T07:57:58.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7fb02dcea700 1 -- start start 2023-10-07T07:57:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7ff886ffd700 1 --2- 172.21.15.7:0/3234461166 >> v2:172.21.15.172:3300/0 conn(0x7ff8880c9af0 0x7ff88815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565821389 0 0) 0x7ff88815a610 con 0x7ff8880c9ec0 2023-10-07T07:57:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff870003570 con 0x7ff8880c9ec0 2023-10-07T07:57:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458588593 0 0) 0x7ff88815a330 con 0x7ff888063230 2023-10-07T07:57:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.862+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff88815a610 con 0x7ff888063230 2023-10-07T07:57:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb02dcea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0280c7630 0x7fb02814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb02dcea700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0281563a0 0x7fb0280b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb02dcea700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb028156770 0x7fb0280c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb02dcea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280c4c50 con 0x7fb0280c7630 2023-10-07T07:57:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb02dcea700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280c4340 con 0x7fb028156770 2023-10-07T07:57:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb02dcea700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02814c230 con 0x7fb0281563a0 2023-10-07T07:57:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1686439525 0 0) 0x7ff88815a610 con 0x7ff888063230 2023-10-07T07:57:58.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 >> v2:172.21.15.172:3300/0 conn(0x7ff8880c9af0 msgr2=0x7ff88815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 --2- 172.21.15.7:0/3234461166 >> v2:172.21.15.172:3300/0 conn(0x7ff8880c9af0 0x7ff88815d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 >> v1:172.21.15.172:6789/0 conn(0x7ff8880c9ec0 legacy=0x7ff888158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88815a8f0 con 0x7ff888063230 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7ff87c001320 con 0x7ff888063230 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 >> v1:172.21.15.7:6789/0 conn(0x7ff888063230 legacy=0x7ff88806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff870004d20 0x7ff870005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8700037b0 0x7ff870003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.863+0000 7ff885ffb700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8700043b0 0x7ff87000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880b03a0 con 0x7ff8700037b0 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87000c300 con 0x7ff8700043b0 2023-10-07T07:57:58.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87000c4c0 con 0x7ff870004d20 2023-10-07T07:57:58.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb027fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb028156770 0x7fb0280c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb027fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb028156770 0x7fb0280c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39534/0 (socket says 172.21.15.7:39534) 2023-10-07T07:57:58.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb027fff700 1 -- 172.21.15.7:0/1561956347 learned_addr learned my addr 172.21.15.7:0/1561956347 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7ff8867fc700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8700037b0 0x7ff870003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb026ffd700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0281563a0 0x7fb0280b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.864+0000 7fb0277fe700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0280c7630 0x7fb02814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7ff8867fc700 1 -- 172.21.15.7:0/3234461166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff870004d20 msgr2=0x7ff870005110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T07:57:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7ff8867fc700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff870004d20 0x7ff870005110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7ff8867fc700 1 -- 172.21.15.7:0/3234461166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8700043b0 msgr2=0x7ff87000bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T07:57:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7fb027fff700 1 -- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0281563a0 msgr2=0x7fb0280b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7fb027fff700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0281563a0 0x7fb0280b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7fb027fff700 1 -- 172.21.15.7:0/1561956347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0280c7630 msgr2=0x7fb02814f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7fb027fff700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0280c7630 0x7fb02814f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7fb027fff700 1 -- 172.21.15.7:0/1561956347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01800a5d0 con 0x7fb028156770 2023-10-07T07:57:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7fb026ffd700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0281563a0 0x7fb0280b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7ff8877fe700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8700043b0 0x7ff87000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7fb027fff700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb028156770 0x7fb0280c50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c890 tx=0x7fb020001670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7fc6d90e4700 1 --2- 172.21.15.7:0/2320174236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d406bd30 0x7fc6d4168880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009e90 tx=0x7fc6cc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79152fc8d797be7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.866+0000 7fb00f7fe700 1 -- 172.21.15.7:0/1561956347 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020012630 con 0x7fb028156770 2023-10-07T07:57:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.866+0000 7fb00f7fe700 1 -- 172.21.15.7:0/1561956347 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb020012790 con 0x7fb028156770 2023-10-07T07:57:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.866+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0280c3e10 con 0x7fb028156770 2023-10-07T07:57:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.866+0000 7fb00f7fe700 1 -- 172.21.15.7:0/1561956347 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020012b50 con 0x7fb028156770 2023-10-07T07:57:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.866+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0280c48e0 con 0x7fb028156770 2023-10-07T07:57:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.868+0000 7fb00f7fe700 1 -- 172.21.15.7:0/1561956347 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb02001a3a0 con 0x7fb028156770 2023-10-07T07:57:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.868+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0280c4e00 con 0x7fb028156770 2023-10-07T07:57:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.868+0000 7fb00f7fe700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb014066f30 0x7fb0140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.869+0000 7fb00f7fe700 1 -- 172.21.15.7:0/1561956347 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb020084760 con 0x7fb028156770 2023-10-07T07:57:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7ff8867fc700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8700043b0 0x7ff87000bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.865+0000 7ff8867fc700 1 -- 172.21.15.7:0/3234461166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88815a8f0 con 0x7ff8700037b0 2023-10-07T07:57:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.866+0000 7ff8867fc700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8700037b0 0x7ff870003c20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00c160 tx=0x7ff87c002d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd947060e26d8267 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.867+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00ca10 con 0x7ff8700037b0 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.867+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c00cb70 con 0x7ff8700037b0 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.867+0000 7ff885ffb700 1 -- 172.21.15.7:0/3234461166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c012420 con 0x7ff8700037b0 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.868+0000 7ff88d4eb700 1 -- 172.21.15.7:0/3234461166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8700037b0 msgr2=0x7ff870003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.868+0000 7ff88d4eb700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8700037b0 0x7ff870003c20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00c160 tx=0x7ff87c002d90 comp rx=0 tx=0).stop 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.869+0000 7ff8877fe700 1 --2- 172.21.15.7:0/3234461166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8700043b0 0x7ff87000bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.869+0000 7ff8877fe700 1 -- 172.21.15.7:0/3234461166 reap_dead start 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.869+0000 7ff8877fe700 1 -- 172.21.15.7:0/3234461166 reap_dead start 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.869+0000 7ff88d4eb700 1 -- 172.21.15.7:0/3234461166 shutdown_connections 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.869+0000 7ff88d4eb700 1 -- 172.21.15.7:0/3234461166 >> 172.21.15.7:0/3234461166 conn(0x7ff8880cd1a0 msgr2=0x7ff8880c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.869+0000 7ff88d4eb700 1 -- 172.21.15.7:0/3234461166 shutdown_connections 2023-10-07T07:57:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.869+0000 7ff88d4eb700 1 -- 172.21.15.7:0/3234461166 wait complete. 2023-10-07T07:57:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.871+0000 7ff88d4eb700 1 Processor -- start 2023-10-07T07:57:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.871+0000 7ff88d4eb700 1 -- start start 2023-10-07T07:57:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff88d4eb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff888158390 0x7ff888151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff88d4eb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff88815cb30 0x7ff88814d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff88d4eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88814db30 0x7ff88815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff88d4eb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88814f8c0 con 0x7ff88814db30 2023-10-07T07:57:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff88d4eb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88815ab80 con 0x7ff888158390 2023-10-07T07:57:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff88d4eb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff888151a60 con 0x7ff88815cb30 2023-10-07T07:57:58.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7fb0277fe700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb014066f30 0x7fb0140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff886ffd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff888158390 0x7ff888151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7fc6d37fe700 1 -- 172.21.15.7:0/2320174236 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00e070 con 0x7fc6d406bd30 2023-10-07T07:57:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7fc6d37fe700 1 -- 172.21.15.7:0/2320174236 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc00bbb0 con 0x7fc6d406bd30 2023-10-07T07:57:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7fc6d37fe700 1 -- 172.21.15.7:0/2320174236 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc012430 con 0x7fc6d406bd30 2023-10-07T07:57:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff886ffd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff888158390 0x7ff888151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39554/0 (socket says 172.21.15.7:39554) 2023-10-07T07:57:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff886ffd700 1 -- 172.21.15.7:0/1947122632 learned_addr learned my addr 172.21.15.7:0/1947122632 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff8877fe700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88814db30 0x7ff88815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.872+0000 7ff8867fc700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff88815cb30 0x7ff88814d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7ff8877fe700 1 -- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff88815cb30 msgr2=0x7ff88814d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7ff8877fe700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff88815cb30 0x7ff88814d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7ff8877fe700 1 -- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff888158390 msgr2=0x7ff888151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7ff8877fe700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff888158390 0x7ff888151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7ff8877fe700 1 -- 172.21.15.7:0/1947122632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88815add0 con 0x7ff88814db30 2023-10-07T07:57:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7ff8877fe700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88814db30 0x7ff88815b8b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ca400 tx=0x7ff878009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7fc6dab47700 1 -- 172.21.15.7:0/2320174236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d406bd30 msgr2=0x7fc6d4168880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7fc6dab47700 1 --2- 172.21.15.7:0/2320174236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d406bd30 0x7fc6d4168880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009e90 tx=0x7fc6cc009780 comp rx=0 tx=0).stop 2023-10-07T07:57:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7ff86effd700 1 -- 172.21.15.7:0/1947122632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878001d00 con 0x7ff88814db30 2023-10-07T07:57:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7fb00f7fe700 1 -- 172.21.15.7:0/1561956347 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0280c4e00 con 0x7fb028156770 2023-10-07T07:57:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7ff86effd700 1 -- 172.21.15.7:0/1947122632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878004b00 con 0x7ff88814db30 2023-10-07T07:57:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.873+0000 7ff86effd700 1 -- 172.21.15.7:0/1947122632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878007190 con 0x7ff88814db30 2023-10-07T07:57:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.876+0000 7f03713ac700 1 -- 172.21.15.7:0/1818572879 shutdown_connections 2023-10-07T07:57:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.874+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88815b0b0 con 0x7ff88814db30 2023-10-07T07:57:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.874+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8881516a0 con 0x7ff88814db30 2023-10-07T07:57:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.876+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88815b5d0 con 0x7ff88814db30 2023-10-07T07:57:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.877+0000 7ff886ffd700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff888158390 0x7ff888151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:57:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.877+0000 7fc6dab47700 1 -- 172.21.15.7:0/2320174236 shutdown_connections 2023-10-07T07:57:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.877+0000 7fc6dab47700 1 --2- 172.21.15.7:0/2320174236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d406bd30 0x7fc6d4168880 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.877+0000 7fc6dab47700 1 -- 172.21.15.7:0/2320174236 >> 172.21.15.7:0/2320174236 conn(0x7fc6d4067ac0 msgr2=0x7fc6d40ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.877+0000 7f03713ac700 1 -- 172.21.15.7:0/1818572879 wait complete. 2023-10-07T07:57:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.879+0000 7f03713ac700 1 Processor -- start 2023-10-07T07:57:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.883+0000 7fc6dab47700 1 -- 172.21.15.7:0/2320174236 shutdown_connections 2023-10-07T07:57:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.883+0000 7ff86effd700 1 -- 172.21.15.7:0/1947122632 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff878007540 con 0x7ff88814db30 2023-10-07T07:57:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.883+0000 7ff86effd700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff874067080 0x7ff874069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.884+0000 7ff86effd700 1 -- 172.21.15.7:0/1947122632 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff878084210 con 0x7ff88814db30 2023-10-07T07:57:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.884+0000 7ff86effd700 1 -- 172.21.15.7:0/1947122632 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff878084600 con 0x7ff88814db30 2023-10-07T07:57:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.885+0000 7fb0277fe700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb014066f30 0x7fb0140693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb01800a5a0 tx=0x7fb01801a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.885+0000 7fc6dab47700 1 -- 172.21.15.7:0/2320174236 wait complete. 2023-10-07T07:57:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.886+0000 7f03713ac700 1 -- start start 2023-10-07T07:57:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.886+0000 7ff886ffd700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff874067080 0x7ff874069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f03713ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f036c140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f03713ac700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f036c13d3c0 0x7f036c141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f03713ac700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036c141a50 0x7f036c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f03713ac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c141ef0 con 0x7f03540052e0 2023-10-07T07:57:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f03713ac700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c13cf30 con 0x7f036c13d3c0 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f03713ac700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c13c360 con 0x7f036c141a50 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f036affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f036c140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f036a7fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f036c13d3c0 0x7f036c141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f036a7fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f036c13d3c0 0x7f036c141510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39558/0 (socket says 172.21.15.7:39558) 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f036a7fc700 1 -- 172.21.15.7:0/1799299976 learned_addr learned my addr 172.21.15.7:0/1799299976 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7f036affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f036c140940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33970/0 (socket says 172.21.15.7:33970) 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.888+0000 7f036b7fe700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036c141a50 0x7f036c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.888+0000 7f036affd700 1 -- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036c141a50 msgr2=0x7f036c15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.888+0000 7f036affd700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036c141a50 0x7f036c15e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.888+0000 7f036affd700 1 -- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f036c13d3c0 msgr2=0x7f036c141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.888+0000 7f036affd700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f036c13d3c0 0x7f036c141510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.888+0000 7f036affd700 1 -- 172.21.15.7:0/1799299976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c13c600 con 0x7f03540052e0 2023-10-07T07:57:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.888+0000 7f036affd700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f036c140940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f036c06f730 tx=0x7f0360001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.888+0000 7f0352ffd700 1 -- 172.21.15.7:0/1799299976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360018360 con 0x7f03540052e0 2023-10-07T07:57:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7f0352ffd700 1 -- 172.21.15.7:0/1799299976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03600184e0 con 0x7f03540052e0 2023-10-07T07:57:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7f0352ffd700 1 -- 172.21.15.7:0/1799299976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360015840 con 0x7f03540052e0 2023-10-07T07:57:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c13c8e0 con 0x7f03540052e0 2023-10-07T07:57:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c0c6df0 con 0x7f03540052e0 2023-10-07T07:57:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.887+0000 7fc6dab47700 1 Processor -- start 2023-10-07T07:57:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.888+0000 7fc6dab47700 1 -- start start 2023-10-07T07:57:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7fc6dab47700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d406bd30 0x7fc6d4158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7fc6dab47700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d415dd80 0x7fc6d4154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7fc6dab47700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d4161830 0x7fc6d4155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7fc6dab47700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4157f30 con 0x7fc6d415dd80 2023-10-07T07:57:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7fc6dab47700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4159430 con 0x7fc6d4161830 2023-10-07T07:57:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.889+0000 7fc6dab47700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41531a0 con 0x7fc6d406bd30 2023-10-07T07:57:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.890+0000 7fc6d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d415dd80 0x7fc6d4154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.890+0000 7fc6d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d415dd80 0x7fc6d4154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33986/0 (socket says 172.21.15.7:33986) 2023-10-07T07:57:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.890+0000 7fc6d3fff700 1 -- 172.21.15.7:0/3500794390 learned_addr learned my addr 172.21.15.7:0/3500794390 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.890+0000 7fc6d88e3700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d406bd30 0x7fc6d4158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.890+0000 7fc6d90e4700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d4161830 0x7fc6d4155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.891+0000 7fc6d90e4700 1 -- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d406bd30 msgr2=0x7fc6d4158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.891+0000 7fc6d90e4700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d406bd30 0x7fc6d4158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.891+0000 7fc6d90e4700 1 -- 172.21.15.7:0/3500794390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d415dd80 msgr2=0x7fc6d4154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.891+0000 7fc6d90e4700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d415dd80 0x7fc6d4154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.891+0000 7fc6d90e4700 1 -- 172.21.15.7:0/3500794390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc009a20 con 0x7fc6d4161830 2023-10-07T07:57:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.891+0000 7fc6d90e4700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d4161830 0x7fc6d4155320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc015000 tx=0x7fc6cc018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.892+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c0052b0 con 0x7f03540052e0 2023-10-07T07:57:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.892+0000 7f0352ffd700 1 -- 172.21.15.7:0/1799299976 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0360015bd0 con 0x7f03540052e0 2023-10-07T07:57:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.893+0000 7f0352ffd700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0358067000 0x7f03580694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.893+0000 7f0352ffd700 1 -- 172.21.15.7:0/1799299976 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0360094be0 con 0x7f03540052e0 2023-10-07T07:57:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.893+0000 7f036a7fc700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0358067000 0x7f03580694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.893+0000 7fc6d0ff9700 1 -- 172.21.15.7:0/3500794390 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc012e10 con 0x7fc6d4161830 2023-10-07T07:57:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.893+0000 7fc6d0ff9700 1 -- 172.21.15.7:0/3500794390 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc01d640 con 0x7fc6d4161830 2023-10-07T07:57:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.893+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d4153480 con 0x7fc6d4161830 2023-10-07T07:57:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.893+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d4158400 con 0x7fc6d4161830 2023-10-07T07:57:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.894+0000 7f036a7fc700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0358067000 0x7f03580694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f036c13d7b0 tx=0x7f035c007ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.895+0000 7ff886ffd700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff874067080 0x7ff874069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff88000e5c0 tx=0x7ff8800077b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.897+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d4153610 con 0x7fc6d4161830 2023-10-07T07:57:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.901+0000 7fc6d0ff9700 1 -- 172.21.15.7:0/3500794390 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc01da00 con 0x7fc6d4161830 2023-10-07T07:57:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.901+0000 7fc6d0ff9700 1 -- 172.21.15.7:0/3500794390 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6cc024070 con 0x7fc6d4161830 2023-10-07T07:57:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.901+0000 7fc6d0ff9700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc6c0067100 0x7fc6c00695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.901+0000 7fc6d0ff9700 1 -- 172.21.15.7:0/3500794390 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc6cc086dc0 con 0x7fc6d4161830 2023-10-07T07:57:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.902+0000 7fc6d0ff9700 1 -- 172.21.15.7:0/3500794390 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6cc08b070 con 0x7fc6d4161830 2023-10-07T07:57:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.903+0000 7fc6d88e3700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc6c0067100 0x7fc6c00695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.903+0000 7fc6d88e3700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc6c0067100 0x7fc6c00695b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c40090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.904+0000 7f0352ffd700 1 -- 172.21.15.7:0/1799299976 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0360061eb0 con 0x7f03540052e0 2023-10-07T07:57:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.918+0000 7f89f3797700 1 Processor -- start 2023-10-07T07:57:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.918+0000 7f89f3797700 1 -- start start 2023-10-07T07:57:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.920+0000 7f89f3797700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89ec066ff0 0x7f89ec0c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.920+0000 7f89f3797700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89ec0b0110 0x7f89ec0c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.920+0000 7f89f3797700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ec068090 con 0x7f89ec147040 2023-10-07T07:57:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.920+0000 7f89f3797700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ec06f050 con 0x7f89ec0b0110 2023-10-07T07:57:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.920+0000 7f89f3797700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ec067480 con 0x7f89ec066ff0 2023-10-07T07:57:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.921+0000 7f89f0d32700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89ec066ff0 0x7f89ec0c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.921+0000 7f89f0d32700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89ec066ff0 0x7f89ec0c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39586/0 (socket says 172.21.15.7:39586) 2023-10-07T07:57:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.921+0000 7f89f0d32700 1 -- 172.21.15.7:0/1426562706 learned_addr learned my addr 172.21.15.7:0/1426562706 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.928+0000 7f89f1d34700 1 --2- 172.21.15.7:0/1426562706 >> v2:172.21.15.7:3300/0 conn(0x7f89ec0b0110 0x7f89ec0c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.928+0000 7f89f0d32700 1 -- 172.21.15.7:0/1426562706 >> v2:172.21.15.7:3300/0 conn(0x7f89ec0b0110 msgr2=0x7f89ec0c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.928+0000 7f89f0d32700 1 --2- 172.21.15.7:0/1426562706 >> v2:172.21.15.7:3300/0 conn(0x7f89ec0b0110 0x7f89ec0c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.928+0000 7f89f0d32700 1 -- 172.21.15.7:0/1426562706 >> v1:172.21.15.7:6789/0 conn(0x7f89ec147040 legacy=0x7f89ec161e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.928+0000 7f89f0d32700 1 -- 172.21.15.7:0/1426562706 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec067760 con 0x7f89ec066ff0 2023-10-07T07:57:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.928+0000 7f89f0d32700 1 --2- 172.21.15.7:0/1426562706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89ec066ff0 0x7f89ec0c98a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89ec063fa0 tx=0x7f89e8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff76bfa79c71a69 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.929+0000 7f89f1d34700 1 --2- 172.21.15.7:0/1426562706 >> v2:172.21.15.7:3300/0 conn(0x7f89ec0b0110 0x7f89ec0c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:57:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.929+0000 7fbb6c7e4700 1 Processor -- start 2023-10-07T07:57:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.929+0000 7fbb6c7e4700 1 -- start start 2023-10-07T07:57:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.930+0000 7f89e3fff700 1 -- 172.21.15.7:0/1426562706 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800b820 con 0x7f89ec066ff0 2023-10-07T07:57:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.930+0000 7f89e3fff700 1 -- 172.21.15.7:0/1426562706 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e800b980 con 0x7f89ec066ff0 2023-10-07T07:57:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.930+0000 7f89e3fff700 1 -- 172.21.15.7:0/1426562706 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800bd00 con 0x7f89ec066ff0 2023-10-07T07:57:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7f89f3797700 1 -- 172.21.15.7:0/1426562706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89ec066ff0 msgr2=0x7f89ec0c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7f89f3797700 1 --2- 172.21.15.7:0/1426562706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89ec066ff0 0x7f89ec0c98a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89ec063fa0 tx=0x7f89e8009250 comp rx=0 tx=0).stop 2023-10-07T07:57:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7fbb6c7e4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb6406dd70 0x7fbb6406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7fbb6c7e4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640c7b30 con 0x7fbb640c6f20 2023-10-07T07:57:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7fbb6c7e4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640c7e10 con 0x7fbb640cc9a0 2023-10-07T07:57:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7fbb6c7e4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6414c950 con 0x7fbb6406dd70 2023-10-07T07:57:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7fbb69d7f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb6406dd70 0x7fbb6406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7fbb6ad81700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbb640c6f20 0x7fbb640c7360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36074/0 (socket says 172.21.15.7:36074) 2023-10-07T07:57:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7fbb69d7f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb6406dd70 0x7fbb6406e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:39592/0 (socket says 172.21.15.7:39592) 2023-10-07T07:57:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.931+0000 7fbb6ad81700 1 -- 172.21.15.7:0/1947683688 learned_addr learned my addr 172.21.15.7:0/1947683688 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.932+0000 7fbb69d7f700 1 -- 172.21.15.7:0/1947683688 >> v1:172.21.15.172:6789/0 conn(0x7fbb640cc9a0 legacy=0x7fbb64156150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.932+0000 7fbb69d7f700 1 -- 172.21.15.7:0/1947683688 >> v1:172.21.15.7:6789/0 conn(0x7fbb640c6f20 legacy=0x7fbb640c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.932+0000 7fbb69d7f700 1 -- 172.21.15.7:0/1947683688 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6415c330 con 0x7fbb6406dd70 2023-10-07T07:57:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.932+0000 7fbb69d7f700 1 --2- 172.21.15.7:0/1947683688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb6406dd70 0x7fbb6406e1f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009e60 tx=0x7fbb58009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=531f660f984ecad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.933+0000 7f89f3797700 1 -- 172.21.15.7:0/1426562706 shutdown_connections 2023-10-07T07:57:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.933+0000 7f89f3797700 1 --2- 172.21.15.7:0/1426562706 >> v2:172.21.15.7:3300/0 conn(0x7f89ec0b0110 0x7f89ec0c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.933+0000 7f89f3797700 1 --2- 172.21.15.7:0/1426562706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89ec066ff0 0x7f89ec0c98a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.933+0000 7f89f3797700 1 -- 172.21.15.7:0/1426562706 >> 172.21.15.7:0/1426562706 conn(0x7f89ec0cd1a0 msgr2=0x7f89ec0c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.933+0000 7fbb6957e700 1 -- 172.21.15.7:0/1947683688 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800e070 con 0x7fbb6406dd70 2023-10-07T07:57:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.933+0000 7fbb6957e700 1 -- 172.21.15.7:0/1947683688 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5800bc80 con 0x7fbb6406dd70 2023-10-07T07:57:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.933+0000 7fbb6957e700 1 -- 172.21.15.7:0/1947683688 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58012430 con 0x7fbb6406dd70 2023-10-07T07:57:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.934+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/1947683688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb6406dd70 msgr2=0x7fbb6406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.934+0000 7fbb6c7e4700 1 --2- 172.21.15.7:0/1947683688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb6406dd70 0x7fbb6406e1f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009e60 tx=0x7fbb58009780 comp rx=0 tx=0).stop 2023-10-07T07:57:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.934+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/1947683688 shutdown_connections 2023-10-07T07:57:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.934+0000 7fbb6c7e4700 1 --2- 172.21.15.7:0/1947683688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb6406dd70 0x7fbb6406e1f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.934+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/1947683688 >> 172.21.15.7:0/1947683688 conn(0x7fbb640cbe90 msgr2=0x7fbb640cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.934+0000 7f89f3797700 1 -- 172.21.15.7:0/1426562706 shutdown_connections 2023-10-07T07:57:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.935+0000 7f89f3797700 1 -- 172.21.15.7:0/1426562706 wait complete. 2023-10-07T07:57:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.937+0000 7f89f3797700 1 Processor -- start 2023-10-07T07:57:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.937+0000 7f89f3797700 1 -- start start 2023-10-07T07:57:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.938+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/1947683688 shutdown_connections 2023-10-07T07:57:58.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.938+0000 7f89f3797700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89ec066ff0 0x7f89ec0c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.938+0000 7f89f3797700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89ec0b0110 0x7f89ec167160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.938+0000 7f89f3797700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89ec147040 0x7f89ec15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.938+0000 7f89f3797700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec167970 con 0x7f89ec0b0110 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.938+0000 7f89f3797700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec165020 con 0x7f89ec147040 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.938+0000 7f89f3797700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec1676a0 con 0x7f89ec066ff0 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.939+0000 7f89f0d32700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89ec0b0110 0x7f89ec167160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.939+0000 7f89f0d32700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89ec0b0110 0x7f89ec167160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33992/0 (socket says 172.21.15.7:33992) 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.939+0000 7f89f0d32700 1 -- 172.21.15.7:0/1164999656 learned_addr learned my addr 172.21.15.7:0/1164999656 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.939+0000 7f89f0d32700 1 -- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89ec066ff0 msgr2=0x7f89ec0c30f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-07T07:57:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.939+0000 7f89f0d32700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89ec066ff0 0x7f89ec0c30f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.939+0000 7f89f0d32700 1 -- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89ec147040 msgr2=0x7f89ec15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.939+0000 7f89f0d32700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89ec147040 0x7f89ec15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.939+0000 7f89f0d32700 1 -- 172.21.15.7:0/1164999656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80094f0 con 0x7f89ec0b0110 2023-10-07T07:57:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.939+0000 7f89f0d32700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89ec0b0110 0x7f89ec167160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f89e80090b0 tx=0x7f89e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.943+0000 7f89e17fa700 1 -- 172.21.15.7:0/1164999656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e801a460 con 0x7f89ec0b0110 2023-10-07T07:57:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.943+0000 7f89e17fa700 1 -- 172.21.15.7:0/1164999656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e801a5c0 con 0x7f89ec0b0110 2023-10-07T07:57:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.943+0000 7f89e17fa700 1 -- 172.21.15.7:0/1164999656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e801a980 con 0x7f89ec0b0110 2023-10-07T07:57:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.943+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ec1646f0 con 0x7f89ec0b0110 2023-10-07T07:57:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.943+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ec164cb0 con 0x7f89ec0b0110 2023-10-07T07:57:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.943+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/1947683688 wait complete. 2023-10-07T07:57:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.945+0000 7fbb6c7e4700 1 Processor -- start 2023-10-07T07:57:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.948+0000 7fbb6c7e4700 1 -- start start 2023-10-07T07:57:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.950+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c80052b0 con 0x7f89ec0b0110 2023-10-07T07:57:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.949+0000 7fbb6c7e4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb6406dd70 0x7fbb6414ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.949+0000 7fbb6c7e4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb640c6f20 0x7fbb640b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.949+0000 7fbb6c7e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb640cc9a0 0x7fbb640c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.949+0000 7fbb6c7e4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6414bf90 con 0x7fbb640cc9a0 2023-10-07T07:57:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.949+0000 7fbb6c7e4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b0340 con 0x7fbb640c6f20 2023-10-07T07:57:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.949+0000 7fbb6c7e4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b0620 con 0x7fbb6406dd70 2023-10-07T07:57:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.950+0000 7fbb6ad81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb640cc9a0 0x7fbb640c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.950+0000 7fbb6ad81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb640cc9a0 0x7fbb640c1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34006/0 (socket says 172.21.15.7:34006) 2023-10-07T07:57:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.950+0000 7fbb6ad81700 1 -- 172.21.15.7:0/582334736 learned_addr learned my addr 172.21.15.7:0/582334736 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.950+0000 7f89e17fa700 1 -- 172.21.15.7:0/1164999656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89e801acf0 con 0x7f89ec0b0110 2023-10-07T07:57:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.950+0000 7fbb6ad81700 1 -- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb6406dd70 msgr2=0x7fbb6414ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.950+0000 7fbb6ad81700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb6406dd70 0x7fbb6414ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.950+0000 7fbb6ad81700 1 -- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb640c6f20 msgr2=0x7fbb640b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.950+0000 7fbb6ad81700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb640c6f20 0x7fbb640b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.951+0000 7fbb6ad81700 1 -- 172.21.15.7:0/582334736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb58009a20 con 0x7fbb640cc9a0 2023-10-07T07:57:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.951+0000 7f89e17fa700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f89d0067050 0x7f89d0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.951+0000 7f89e17fa700 1 -- 172.21.15.7:0/1164999656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f89e8085e20 con 0x7f89ec0b0110 2023-10-07T07:57:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.951+0000 7fbb6ad81700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb640cc9a0 0x7fbb640c1c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c7c0 tx=0x7fbb5c007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.953+0000 7f89f1533700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f89d0067050 0x7f89d0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.951+0000 7fbb627fc700 1 -- 172.21.15.7:0/582334736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c005b30 con 0x7fbb640cc9a0 2023-10-07T07:57:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.952+0000 7fbb627fc700 1 -- 172.21.15.7:0/582334736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c005cb0 con 0x7fbb640cc9a0 2023-10-07T07:57:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.952+0000 7fbb627fc700 1 -- 172.21.15.7:0/582334736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c007320 con 0x7fbb640cc9a0 2023-10-07T07:57:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.952+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb640b0900 con 0x7fbb640cc9a0 2023-10-07T07:57:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.952+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb640b0ec0 con 0x7fbb640cc9a0 2023-10-07T07:57:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.957+0000 7f89f1533700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f89d0067050 0x7f89d0069500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009740 tx=0x7f89dc006c60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.957+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb640b10e0 con 0x7fbb640cc9a0 2023-10-07T07:57:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.957+0000 7f61ae1f5700 1 Processor -- start 2023-10-07T07:57:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.957+0000 7f61ae1f5700 1 -- start start 2023-10-07T07:57:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.959+0000 7f61ae1f5700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61a81407d0 0x7f61a80aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.959+0000 7f61ae1f5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a8140ba0 0x7f61a806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.959+0000 7f61ae1f5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a8069c70 con 0x7f61a806d7f0 2023-10-07T07:57:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.959+0000 7f61ae1f5700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a8069380 con 0x7f61a81407d0 2023-10-07T07:57:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.959+0000 7f61ae1f5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a806dc70 con 0x7f61a8140ba0 2023-10-07T07:57:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.959+0000 7f61a7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f61a806d7f0 0x7f61a806a100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36076/0 (socket says 172.21.15.7:36076) 2023-10-07T07:57:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.959+0000 7f61a7fff700 1 -- 172.21.15.7:0/746804520 learned_addr learned my addr 172.21.15.7:0/746804520 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:57:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.961+0000 7f61a6ffd700 1 --2- 172.21.15.7:0/746804520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a8140ba0 0x7f61a806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.961+0000 7f89e17fa700 1 -- 172.21.15.7:0/1164999656 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89e8053070 con 0x7f89ec0b0110 2023-10-07T07:57:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.961+0000 7f61a6ffd700 1 -- 172.21.15.7:0/746804520 >> v2:172.21.15.172:3300/0 conn(0x7f61a81407d0 msgr2=0x7f61a80aec50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T07:57:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.961+0000 7f61a6ffd700 1 --2- 172.21.15.7:0/746804520 >> v2:172.21.15.172:3300/0 conn(0x7f61a81407d0 0x7f61a80aec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.961+0000 7f61a6ffd700 1 -- 172.21.15.7:0/746804520 >> v1:172.21.15.7:6789/0 conn(0x7f61a806d7f0 legacy=0x7f61a806a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.962+0000 7f61a6ffd700 1 -- 172.21.15.7:0/746804520 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a8069620 con 0x7f61a8140ba0 2023-10-07T07:57:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.963+0000 7fbb627fc700 1 -- 172.21.15.7:0/582334736 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb5c007690 con 0x7fbb640cc9a0 2023-10-07T07:57:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.964+0000 7fbb627fc700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb48067000 0x7fbb480694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.964+0000 7fbb627fc700 1 -- 172.21.15.7:0/582334736 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb5c083f30 con 0x7fbb640cc9a0 2023-10-07T07:57:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.964+0000 7fbb6a580700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb48067000 0x7fbb480694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.965+0000 7fbb6a580700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb48067000 0x7fbb480694b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb6406ed00 tx=0x7fbb50006c40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.966+0000 7f61a6ffd700 1 --2- 172.21.15.7:0/746804520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a8140ba0 0x7f61a806d2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61980098e0 tx=0x7f61980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b65723dd4cb2add server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.966+0000 7f61a67fc700 1 -- 172.21.15.7:0/746804520 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198009e80 con 0x7f61a8140ba0 2023-10-07T07:57:58.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.966+0000 7f61a67fc700 1 -- 172.21.15.7:0/746804520 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619800b860 con 0x7f61a8140ba0 2023-10-07T07:57:58.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.966+0000 7f61a67fc700 1 -- 172.21.15.7:0/746804520 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800bc20 con 0x7f61a8140ba0 2023-10-07T07:57:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.967+0000 7f61ae1f5700 1 -- 172.21.15.7:0/746804520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a8140ba0 msgr2=0x7f61a806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.967+0000 7f61ae1f5700 1 --2- 172.21.15.7:0/746804520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a8140ba0 0x7f61a806d2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61980098e0 tx=0x7f61980091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.967+0000 7f61ae1f5700 1 -- 172.21.15.7:0/746804520 shutdown_connections 2023-10-07T07:57:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.967+0000 7f61ae1f5700 1 --2- 172.21.15.7:0/746804520 >> v2:172.21.15.172:3300/0 conn(0x7f61a81407d0 0x7f61a80aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.967+0000 7f61ae1f5700 1 --2- 172.21.15.7:0/746804520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a8140ba0 0x7f61a806d2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.967+0000 7f61ae1f5700 1 -- 172.21.15.7:0/746804520 >> 172.21.15.7:0/746804520 conn(0x7f61a8067a80 msgr2=0x7f61a80cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.968+0000 7f61ae1f5700 1 -- 172.21.15.7:0/746804520 shutdown_connections 2023-10-07T07:57:58.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.968+0000 7f61ae1f5700 1 -- 172.21.15.7:0/746804520 wait complete. 2023-10-07T07:57:58.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.970+0000 7f61ae1f5700 1 Processor -- start 2023-10-07T07:57:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.970+0000 7f61ae1f5700 1 -- start start 2023-10-07T07:57:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.971+0000 7fbb627fc700 1 -- 172.21.15.7:0/582334736 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb5c051200 con 0x7fbb640cc9a0 2023-10-07T07:57:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.971+0000 7f61ae1f5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a806d7f0 0x7f61a80c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.971+0000 7f61ae1f5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a81407d0 0x7f61a81768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.971+0000 7f61ae1f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61a8140ba0 0x7f61a80adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.971+0000 7f61ae1f5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80af640 con 0x7f61a8140ba0 2023-10-07T07:57:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.971+0000 7f61ae1f5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80b04a0 con 0x7f61a81407d0 2023-10-07T07:57:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.971+0000 7f61ae1f5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80c3e90 con 0x7f61a806d7f0 2023-10-07T07:57:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.972+0000 7f61a77fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a806d7f0 0x7f61a80c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.972+0000 7f61a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61a8140ba0 0x7f61a80adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.972+0000 7f61a77fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a806d7f0 0x7f61a80c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:32818/0 (socket says 172.21.15.7:32818) 2023-10-07T07:57:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.972+0000 7f61a77fe700 1 -- 172.21.15.7:0/372117107 learned_addr learned my addr 172.21.15.7:0/372117107 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.972+0000 7f61a7fff700 1 -- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a806d7f0 msgr2=0x7f61a80c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.972+0000 7f61a7fff700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a806d7f0 0x7f61a80c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.972+0000 7f61a7fff700 1 -- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a81407d0 msgr2=0x7f61a81768a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-07T07:57:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.972+0000 7f61a7fff700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a81407d0 0x7f61a81768a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.972+0000 7f61a7fff700 1 -- 172.21.15.7:0/372117107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6198009420 con 0x7f61a8140ba0 2023-10-07T07:57:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.973+0000 7f61a7fff700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61a8140ba0 0x7f61a80adac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f61a000cb90 tx=0x7f61a00019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.973+0000 7f618f7fe700 1 -- 172.21.15.7:0/372117107 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a000a910 con 0x7f61a8140ba0 2023-10-07T07:57:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.973+0000 7f618f7fe700 1 -- 172.21.15.7:0/372117107 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a000aa70 con 0x7f61a8140ba0 2023-10-07T07:57:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.973+0000 7f618f7fe700 1 -- 172.21.15.7:0/372117107 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a000ae30 con 0x7f61a8140ba0 2023-10-07T07:57:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.973+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a80af890 con 0x7f61a8140ba0 2023-10-07T07:57:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.973+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a80afe50 con 0x7f61a8140ba0 2023-10-07T07:57:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.975+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a80b0610 con 0x7f61a8140ba0 2023-10-07T07:57:58.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.981+0000 7f618f7fe700 1 -- 172.21.15.7:0/372117107 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61a0007300 con 0x7f61a8140ba0 2023-10-07T07:57:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.982+0000 7f618f7fe700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6194066f30 0x7f61940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:58.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.982+0000 7f618f7fe700 1 -- 172.21.15.7:0/372117107 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61a0085430 con 0x7f61a8140ba0 2023-10-07T07:57:58.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.983+0000 7f61a77fe700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6194066f30 0x7f61940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:58.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.983+0000 7f61a77fe700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6194066f30 0x7f61940693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f619c009aa0 tx=0x7f619c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:58.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:58.987+0000 7f618f7fe700 1 -- 172.21.15.7:0/372117107 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61a0052700 con 0x7f61a8140ba0 2023-10-07T07:57:59.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.272+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb02806d290 con 0x7fb028156770 2023-10-07T07:57:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.277+0000 7fb00f7fe700 1 -- 172.21.15.7:0/1561956347 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb020051930 con 0x7fb028156770 2023-10-07T07:57:59.280 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-10-07T07:57:59.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb014066f30 msgr2=0x7fb0140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb014066f30 0x7fb0140693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb01800a5a0 tx=0x7fb01801a000 comp rx=0 tx=0).stop 2023-10-07T07:57:59.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb028156770 msgr2=0x7fb0280c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb028156770 0x7fb0280c50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c890 tx=0x7fb020001670 comp rx=0 tx=0).stop 2023-10-07T07:57:59.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 shutdown_connections 2023-10-07T07:57:59.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb014066f30 0x7fb0140693e0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb01800a5a0 tx=0x7fb01801a000 comp rx=0 tx=0).stop 2023-10-07T07:57:59.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb028156770 0x7fb0280c50e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0280c7630 0x7fb02814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 --2- 172.21.15.7:0/1561956347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0281563a0 0x7fb0280b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 >> 172.21.15.7:0/1561956347 conn(0x7fb0280cbe90 msgr2=0x7fb0280698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 shutdown_connections 2023-10-07T07:57:59.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.281+0000 7fb02dcea700 1 -- 172.21.15.7:0/1561956347 wait complete. 2023-10-07T07:57:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.299+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7678005560 con 0x7f769806dac0 2023-10-07T07:57:59.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2023-10-07T07:57:59.302 DEBUG:teuthology.parallel:result is None 2023-10-07T07:57:59.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.302+0000 7f767effd700 1 -- 172.21.15.7:0/2346079269 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f769005e6e0 con 0x7f769806dac0 2023-10-07T07:57:59.302 INFO:teuthology.orchestra.run.smithi007.stdout:55834574852 2023-10-07T07:57:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.306+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7684067080 msgr2=0x7f7684069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.306+0000 7f769d01c700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7684067080 0x7f7684069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7698087aa0 tx=0x7f768800c040 comp rx=0 tx=0).stop 2023-10-07T07:57:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.306+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769806dac0 msgr2=0x7f7698160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.306+0000 7f769d01c700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769806dac0 0x7f7698160a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7698087f20 tx=0x7f769001b000 comp rx=0 tx=0).stop 2023-10-07T07:57:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.306+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f034c005560 con 0x7f03540052e0 2023-10-07T07:57:59.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.307+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 shutdown_connections 2023-10-07T07:57:59.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.307+0000 7f769d01c700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7684067080 0x7f7684069530 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.307+0000 7f769d01c700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76980c1ee0 0x7f76980c2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.307+0000 7f769d01c700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769806dac0 0x7f7698160a90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.307+0000 7f769d01c700 1 --2- 172.21.15.7:0/2346079269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76981661e0 0x7f769817a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.307+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 >> 172.21.15.7:0/2346079269 conn(0x7f76980cbe90 msgr2=0x7f76980ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.307+0000 7f0352ffd700 1 -- 172.21.15.7:0/1799299976 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0360029070 con 0x7f03540052e0 2023-10-07T07:57:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.307+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 shutdown_connections 2023-10-07T07:57:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.307+0000 7f769d01c700 1 -- 172.21.15.7:0/2346079269 wait complete. 2023-10-07T07:57:59.309 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0358067000 msgr2=0x7f03580694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0358067000 0x7f03580694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f036c13d7b0 tx=0x7f035c007ed0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 msgr2=0x7f036c140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f036c140940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f036c06f730 tx=0x7f0360001a00 comp rx=0 tx=0).stop 2023-10-07T07:57:59.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 shutdown_connections 2023-10-07T07:57:59.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0358067000 0x7f03580694b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f036c141a50 0x7f036c15e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03540052e0 0x7f036c140940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 --2- 172.21.15.7:0/1799299976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f036c13d3c0 0x7f036c141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 >> 172.21.15.7:0/1799299976 conn(0x7f036c068bd0 msgr2=0x7f036c06ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.311+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 shutdown_connections 2023-10-07T07:57:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.312+0000 7f03713ac700 1 -- 172.21.15.7:0/1799299976 wait complete. 2023-10-07T07:57:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.313+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc6d40716c0 con 0x7fc6d4161830 2023-10-07T07:57:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.314+0000 7fc6d0ff9700 1 -- 172.21.15.7:0/3500794390 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc6cc057bd0 con 0x7fc6d4161830 2023-10-07T07:57:59.315 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.315+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff88815b5d0 con 0x7ff88814db30 2023-10-07T07:57:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.315+0000 7ff86effd700 1 -- 172.21.15.7:0/1947122632 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff88815b5d0 con 0x7ff88814db30 2023-10-07T07:57:59.317 INFO:teuthology.orchestra.run.smithi007.stdout:55834574852 2023-10-07T07:57:59.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc6c0067100 msgr2=0x7fc6c00695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc6c0067100 0x7fc6c00695b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c40090f0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d4161830 msgr2=0x7fc6d4155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d4161830 0x7fc6d4155320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc015000 tx=0x7fc6cc018000 comp rx=0 tx=0).stop 2023-10-07T07:57:59.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 shutdown_connections 2023-10-07T07:57:59.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc6c0067100 0x7fc6c00695b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d4161830 0x7fc6d4155320 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6d406bd30 0x7fc6d4158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 --2- 172.21.15.7:0/3500794390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d415dd80 0x7fc6d4154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 >> 172.21.15.7:0/3500794390 conn(0x7fc6d4067ac0 msgr2=0x7fc6d40cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff874067080 msgr2=0x7ff874069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff874067080 0x7ff874069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff88000e5c0 tx=0x7ff8800077b0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88814db30 msgr2=0x7ff88815b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88814db30 0x7ff88815b8b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ca400 tx=0x7ff878009040 comp rx=0 tx=0).stop 2023-10-07T07:57:59.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 shutdown_connections 2023-10-07T07:57:59.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff874067080 0x7ff874069530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88814db30 0x7ff88815b8b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff888158390 0x7ff888151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 --2- 172.21.15.7:0/1947122632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff88815cb30 0x7ff88814d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.319+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 >> 172.21.15.7:0/1947122632 conn(0x7ff8880cd1a0 msgr2=0x7ff888151e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.320+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 shutdown_connections 2023-10-07T07:57:59.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.320+0000 7fc6dab47700 1 -- 172.21.15.7:0/3500794390 wait complete. 2023-10-07T07:57:59.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.320+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 shutdown_connections 2023-10-07T07:57:59.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.320+0000 7ff88d4eb700 1 -- 172.21.15.7:0/1947122632 wait complete. 2023-10-07T07:57:59.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2023-10-07T07:57:59.330 DEBUG:teuthology.parallel:result is None 2023-10-07T07:57:59.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-10-07T07:57:59.335 DEBUG:teuthology.parallel:result is None 2023-10-07T07:57:59.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-07T07:57:59.344 DEBUG:teuthology.parallel:result is None 2023-10-07T07:57:59.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2023-10-07T07:57:59.346 DEBUG:teuthology.parallel:result is None 2023-10-07T07:57:59.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.358+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbb640716c0 con 0x7fbb640cc9a0 2023-10-07T07:57:59.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.359+0000 7fbb627fc700 1 -- 172.21.15.7:0/582334736 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb5c017090 con 0x7fbb640cc9a0 2023-10-07T07:57:59.359 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb48067000 msgr2=0x7fbb480694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb48067000 0x7fbb480694b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb6406ed00 tx=0x7fbb50006c40 comp rx=0 tx=0).stop 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb640cc9a0 msgr2=0x7fbb640c1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb640cc9a0 0x7fbb640c1c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c7c0 tx=0x7fbb5c007eb0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 shutdown_connections 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb48067000 0x7fbb480694b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb640cc9a0 0x7fbb640c1c40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb6406dd70 0x7fbb6414ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 --2- 172.21.15.7:0/582334736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb640c6f20 0x7fbb640b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 >> 172.21.15.7:0/582334736 conn(0x7fbb640cbe90 msgr2=0x7fbb64069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 shutdown_connections 2023-10-07T07:57:59.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.361+0000 7fbb6c7e4700 1 -- 172.21.15.7:0/582334736 wait complete. 2023-10-07T07:57:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.365+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f89c8005560 con 0x7f89ec0b0110 2023-10-07T07:57:59.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.366+0000 7f89e17fa700 1 -- 172.21.15.7:0/1164999656 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f89e8020020 con 0x7f89ec0b0110 2023-10-07T07:57:59.366 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f89d0067050 msgr2=0x7f89d0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f89d0067050 0x7f89d0069500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009740 tx=0x7f89dc006c60 comp rx=0 tx=0).stop 2023-10-07T07:57:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89ec0b0110 msgr2=0x7f89ec167160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89ec0b0110 0x7f89ec167160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f89e80090b0 tx=0x7f89e8016000 comp rx=0 tx=0).stop 2023-10-07T07:57:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 shutdown_connections 2023-10-07T07:57:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f89d0067050 0x7f89d0069500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89ec147040 0x7f89ec15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89ec066ff0 0x7f89ec0c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 --2- 172.21.15.7:0/1164999656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89ec0b0110 0x7f89ec167160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 >> 172.21.15.7:0/1164999656 conn(0x7f89ec0cd1a0 msgr2=0x7f89ec135aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 shutdown_connections 2023-10-07T07:57:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.368+0000 7f89f3797700 1 -- 172.21.15.7:0/1164999656 wait complete. 2023-10-07T07:57:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.375+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f61a80716c0 con 0x7f61a8140ba0 2023-10-07T07:57:59.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-07T07:57:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.376+0000 7f618f7fe700 1 -- 172.21.15.7:0/372117107 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f61a0055d20 con 0x7f61a8140ba0 2023-10-07T07:57:59.377 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-10-07T07:57:59.377 DEBUG:teuthology.parallel:result is None 2023-10-07T07:57:59.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6194066f30 msgr2=0x7f61940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6194066f30 0x7f61940693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f619c009aa0 tx=0x7f619c008040 comp rx=0 tx=0).stop 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61a8140ba0 msgr2=0x7f61a80adac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61a8140ba0 0x7f61a80adac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f61a000cb90 tx=0x7f61a00019a0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 shutdown_connections 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6194066f30 0x7f61940693e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61a8140ba0 0x7f61a80adac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a806d7f0 0x7f61a80c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 --2- 172.21.15.7:0/372117107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a81407d0 0x7f61a81768a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 >> 172.21.15.7:0/372117107 conn(0x7f61a8067a80 msgr2=0x7f61a806a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 shutdown_connections 2023-10-07T07:57:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.378+0000 7f61ae1f5700 1 -- 172.21.15.7:0/372117107 wait complete. 2023-10-07T07:57:59.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-10-07T07:57:59.384 DEBUG:teuthology.parallel:result is None 2023-10-07T07:57:59.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-07T07:57:59.392 DEBUG:teuthology.parallel:result is None 2023-10-07T07:57:59.392 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-07T07:57:59.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T07:57:59.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.567+0000 7ff430e22700 1 Processor -- start 2023-10-07T07:57:59.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.567+0000 7ff430e22700 1 -- start start 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.568+0000 7ff430e22700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff42c1570a0 0x7ff42c0c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.568+0000 7ff430e22700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c157470 0x7ff42c06b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.568+0000 7ff430e22700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42c0c3b80 con 0x7ff42c06bb60 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.568+0000 7ff430e22700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42c06bfe0 con 0x7ff42c1570a0 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.568+0000 7ff430e22700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0c2f70 con 0x7ff42c157470 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff429d9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c157470 0x7ff42c06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff42a59c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff42c1570a0 0x7ff42c0c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff429d9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c157470 0x7ff42c06b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:47608/0 (socket says 172.21.15.7:47608) 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff429d9b700 1 -- 172.21.15.7:0/3793429296 learned_addr learned my addr 172.21.15.7:0/3793429296 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff42a59c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff42c1570a0 0x7ff42c0c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:48276/0 (socket says 172.21.15.7:48276) 2023-10-07T07:57:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff42ad9d700 1 --1- 172.21.15.7:0/3793429296 >> v1:172.21.15.172:6789/0 conn(0x7ff42c06bb60 0x7ff42c06cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:46044/0 (socket says 172.21.15.7:46044) 2023-10-07T07:57:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff42a59c700 1 -- 172.21.15.7:0/3793429296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c157470 msgr2=0x7ff42c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff42a59c700 1 --2- 172.21.15.7:0/3793429296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c157470 0x7ff42c06b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff42a59c700 1 -- 172.21.15.7:0/3793429296 >> v1:172.21.15.172:6789/0 conn(0x7ff42c06bb60 legacy=0x7ff42c06cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff42a59c700 1 -- 172.21.15.7:0/3793429296 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c0c3250 con 0x7ff42c1570a0 2023-10-07T07:57:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff429d9b700 1 --2- 172.21.15.7:0/3793429296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c157470 0x7ff42c06b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:57:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.569+0000 7ff42a59c700 1 --2- 172.21.15.7:0/3793429296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff42c1570a0 0x7ff42c0c8800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0098e0 tx=0x7ff41c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35c7cb2548b5a3ce server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff42959a700 1 -- 172.21.15.7:0/3793429296 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00b640 con 0x7ff42c1570a0 2023-10-07T07:57:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff42959a700 1 -- 172.21.15.7:0/3793429296 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41c00b7a0 con 0x7ff42c1570a0 2023-10-07T07:57:59.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff42959a700 1 -- 172.21.15.7:0/3793429296 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00bb40 con 0x7ff42c1570a0 2023-10-07T07:57:59.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff430e22700 1 -- 172.21.15.7:0/3793429296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff42c1570a0 msgr2=0x7ff42c0c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff430e22700 1 --2- 172.21.15.7:0/3793429296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff42c1570a0 0x7ff42c0c8800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0098e0 tx=0x7ff41c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff430e22700 1 -- 172.21.15.7:0/3793429296 shutdown_connections 2023-10-07T07:57:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff430e22700 1 --2- 172.21.15.7:0/3793429296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff42c1570a0 0x7ff42c0c8800 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff430e22700 1 --2- 172.21.15.7:0/3793429296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c157470 0x7ff42c06b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff430e22700 1 -- 172.21.15.7:0/3793429296 >> 172.21.15.7:0/3793429296 conn(0x7ff42c067ac0 msgr2=0x7ff42c0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff430e22700 1 -- 172.21.15.7:0/3793429296 shutdown_connections 2023-10-07T07:57:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.570+0000 7ff430e22700 1 -- 172.21.15.7:0/3793429296 wait complete. 2023-10-07T07:57:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.572+0000 7ff430e22700 1 Processor -- start 2023-10-07T07:57:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.572+0000 7ff430e22700 1 -- start start 2023-10-07T07:57:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.572+0000 7ff430e22700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c06bb60 0x7ff42c136c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.572+0000 7ff430e22700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff42c1570a0 0x7ff42c137db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.572+0000 7ff430e22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff42c157470 0x7ff42c0c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.572+0000 7ff430e22700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c1382f0 con 0x7ff42c157470 2023-10-07T07:57:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.572+0000 7ff430e22700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0c5590 con 0x7ff42c1570a0 2023-10-07T07:57:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.572+0000 7ff430e22700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0c5e30 con 0x7ff42c06bb60 2023-10-07T07:57:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.572+0000 7ff42ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff42c157470 0x7ff42c0c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff42ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff42c157470 0x7ff42c0c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47860/0 (socket says 172.21.15.7:47860) 2023-10-07T07:57:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff42ad9d700 1 -- 172.21.15.7:0/871313117 learned_addr learned my addr 172.21.15.7:0/871313117 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff42a59c700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c06bb60 0x7ff42c136c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff429d9b700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff42c1570a0 0x7ff42c137db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff42ad9d700 1 -- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c06bb60 msgr2=0x7ff42c136c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff42ad9d700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c06bb60 0x7ff42c136c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff42ad9d700 1 -- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff42c1570a0 msgr2=0x7ff42c137db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff42ad9d700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff42c1570a0 0x7ff42c137db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff42ad9d700 1 -- 172.21.15.7:0/871313117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200094f0 con 0x7ff42c157470 2023-10-07T07:57:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff42ad9d700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff42c157470 0x7ff42c0c62c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff42400cd30 tx=0x7ff424001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff41a7fc700 1 -- 172.21.15.7:0/871313117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400ce90 con 0x7ff42c157470 2023-10-07T07:57:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff41a7fc700 1 -- 172.21.15.7:0/871313117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff424005df0 con 0x7ff42c157470 2023-10-07T07:57:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c009420 con 0x7ff42c157470 2023-10-07T07:57:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff41a7fc700 1 -- 172.21.15.7:0/871313117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400aaa0 con 0x7ff42c157470 2023-10-07T07:57:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.573+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42c0c57e0 con 0x7ff42c157470 2023-10-07T07:57:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.574+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42c0716c0 con 0x7ff42c157470 2023-10-07T07:57:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.575+0000 7ff41a7fc700 1 -- 172.21.15.7:0/871313117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4240070f0 con 0x7ff42c157470 2023-10-07T07:57:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.575+0000 7ff41a7fc700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff40c066f30 0x7ff40c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.575+0000 7ff41a7fc700 1 -- 172.21.15.7:0/871313117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff424084470 con 0x7ff42c157470 2023-10-07T07:57:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.575+0000 7ff42a59c700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff40c066f30 0x7ff40c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.576+0000 7ff42a59c700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff40c066f30 0x7ff40c0693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0093f0 tx=0x7ff41c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:59.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.579+0000 7ff41a7fc700 1 -- 172.21.15.7:0/871313117 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff424051760 con 0x7ff42c157470 2023-10-07T07:57:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.776+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff42c0c9a70 con 0x7ff40c066f30 2023-10-07T07:57:59.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.783+0000 7ff41a7fc700 1 -- 172.21.15.7:0/871313117 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264572 (secure 0 0 0) 0x7ff42c0c9a70 con 0x7ff40c066f30 2023-10-07T07:57:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:57:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff40c066f30 msgr2=0x7ff40c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff40c066f30 0x7ff40c0693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0093f0 tx=0x7ff41c017000 comp rx=0 tx=0).stop 2023-10-07T07:57:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff42c157470 msgr2=0x7ff42c0c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff42c157470 0x7ff42c0c62c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff42400cd30 tx=0x7ff424001b10 comp rx=0 tx=0).stop 2023-10-07T07:57:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 shutdown_connections 2023-10-07T07:57:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff40c066f30 0x7ff40c0693e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff42c157470 0x7ff42c0c62c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff42c06bb60 0x7ff42c136c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 --2- 172.21.15.7:0/871313117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff42c1570a0 0x7ff42c137db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 >> 172.21.15.7:0/871313117 conn(0x7ff42c067ac0 msgr2=0x7ff42c06cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 shutdown_connections 2023-10-07T07:57:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.787+0000 7ff430e22700 1 -- 172.21.15.7:0/871313117 wait complete. 2023-10-07T07:57:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T07:57:59.801 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-07T07:57:58.026975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381224,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593496,"statfs":{"total":773094113280,"available":770655739904,"internally_reserved":0,"allocated":2990080,"data_stored":1787852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001987"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733747+0000","last_change":"2023-10-07T07:57:50.730298+0000","last_active":"2023-10-07T07:57:50.733747+0000","last_peered":"2023-10-07T07:57:50.733747+0000","last_clean":"2023-10-07T07:57:50.733747+0000","last_became_active":"2023-10-07T07:57:50.729981+0000","last_became_peered":"2023-10-07T07:57:50.729981+0000","last_unstale":"2023-10-07T07:57:50.733747+0000","last_undegraded":"2023-10-07T07:57:50.733747+0000","last_fullsized":"2023-10-07T07:57:50.733747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:13:37.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734352+0000","last_change":"2023-10-07T07:57:50.726555+0000","last_active":"2023-10-07T07:57:50.734352+0000","last_peered":"2023-10-07T07:57:50.734352+0000","last_clean":"2023-10-07T07:57:50.734352+0000","last_became_active":"2023-10-07T07:57:50.726036+0000","last_became_peered":"2023-10-07T07:57:50.726036+0000","last_unstale":"2023-10-07T07:57:50.734352+0000","last_undegraded":"2023-10-07T07:57:50.734352+0000","last_fullsized":"2023-10-07T07:57:50.734352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:03:06.838727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726644+0000","last_change":"2023-10-07T07:57:50.722937+0000","last_active":"2023-10-07T07:57:50.726644+0000","last_peered":"2023-10-07T07:57:50.726644+0000","last_clean":"2023-10-07T07:57:50.726644+0000","last_became_active":"2023-10-07T07:57:50.722662+0000","last_became_peered":"2023-10-07T07:57:50.722662+0000","last_unstale":"2023-10-07T07:57:50.726644+0000","last_undegraded":"2023-10-07T07:57:50.726644+0000","last_fullsized":"2023-10-07T07:57:50.726644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:15:16.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741568+0000","last_change":"2023-10-07T07:57:50.725792+0000","last_active":"2023-10-07T07:57:50.741568+0000","last_peered":"2023-10-07T07:57:50.741568+0000","last_clean":"2023-10-07T07:57:50.741568+0000","last_became_active":"2023-10-07T07:57:50.725545+0000","last_became_peered":"2023-10-07T07:57:50.725545+0000","last_unstale":"2023-10-07T07:57:50.741568+0000","last_undegraded":"2023-10-07T07:57:50.741568+0000","last_fullsized":"2023-10-07T07:57:50.741568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739501+0000","last_change":"2023-10-07T07:57:50.737070+0000","last_active":"2023-10-07T07:57:50.739501+0000","last_peered":"2023-10-07T07:57:50.739501+0000","last_clean":"2023-10-07T07:57:50.739501+0000","last_became_active":"2023-10-07T07:57:50.734411+0000","last_became_peered":"2023-10-07T07:57:50.734411+0000","last_unstale":"2023-10-07T07:57:50.739501+0000","last_undegraded":"2023-10-07T07:57:50.739501+0000","last_fullsized":"2023-10-07T07:57:50.739501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:03:23.294266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740242+0000","last_change":"2023-10-07T07:57:50.719875+0000","last_active":"2023-10-07T07:57:50.740242+0000","last_peered":"2023-10-07T07:57:50.740242+0000","last_clean":"2023-10-07T07:57:50.740242+0000","last_became_active":"2023-10-07T07:57:50.719322+0000","last_became_peered":"2023-10-07T07:57:50.719322+0000","last_unstale":"2023-10-07T07:57:50.740242+0000","last_undegraded":"2023-10-07T07:57:50.740242+0000","last_fullsized":"2023-10-07T07:57:50.740242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732962+0000","last_change":"2023-10-07T07:57:50.728894+0000","last_active":"2023-10-07T07:57:50.732962+0000","last_peered":"2023-10-07T07:57:50.732962+0000","last_clean":"2023-10-07T07:57:50.732962+0000","last_became_active":"2023-10-07T07:57:50.726313+0000","last_became_peered":"2023-10-07T07:57:50.726313+0000","last_unstale":"2023-10-07T07:57:50.732962+0000","last_undegraded":"2023-10-07T07:57:50.732962+0000","last_fullsized":"2023-10-07T07:57:50.732962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.736796+0000","last_change":"2023-10-07T07:57:50.730368+0000","last_active":"2023-10-07T07:57:50.736796+0000","last_peered":"2023-10-07T07:57:50.736796+0000","last_clean":"2023-10-07T07:57:50.736796+0000","last_became_active":"2023-10-07T07:57:50.730094+0000","last_became_peered":"2023-10-07T07:57:50.730094+0000","last_unstale":"2023-10-07T07:57:50.736796+0000","last_undegraded":"2023-10-07T07:57:50.736796+0000","last_fullsized":"2023-10-07T07:57:50.736796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:53:49.221151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734017+0000","last_change":"2023-10-07T07:57:50.726948+0000","last_active":"2023-10-07T07:57:50.734017+0000","last_peered":"2023-10-07T07:57:50.734017+0000","last_clean":"2023-10-07T07:57:50.734017+0000","last_became_active":"2023-10-07T07:57:50.726622+0000","last_became_peered":"2023-10-07T07:57:50.726622+0000","last_unstale":"2023-10-07T07:57:50.734017+0000","last_undegraded":"2023-10-07T07:57:50.734017+0000","last_fullsized":"2023-10-07T07:57:50.734017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:38:53.980034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.745995+0000","last_change":"2023-10-07T07:57:50.745365+0000","last_active":"2023-10-07T07:57:50.745995+0000","last_peered":"2023-10-07T07:57:50.745995+0000","last_clean":"2023-10-07T07:57:50.745995+0000","last_became_active":"2023-10-07T07:57:50.745061+0000","last_became_peered":"2023-10-07T07:57:50.745061+0000","last_unstale":"2023-10-07T07:57:50.745995+0000","last_undegraded":"2023-10-07T07:57:50.745995+0000","last_fullsized":"2023-10-07T07:57:50.745995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:09:38.557090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733023+0000","last_change":"2023-10-07T07:57:50.729317+0000","last_active":"2023-10-07T07:57:50.733023+0000","last_peered":"2023-10-07T07:57:50.733023+0000","last_clean":"2023-10-07T07:57:50.733023+0000","last_became_active":"2023-10-07T07:57:50.726065+0000","last_became_peered":"2023-10-07T07:57:50.726065+0000","last_unstale":"2023-10-07T07:57:50.733023+0000","last_undegraded":"2023-10-07T07:57:50.733023+0000","last_fullsized":"2023-10-07T07:57:50.733023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:59:47.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735217+0000","last_change":"2023-10-07T07:57:50.734224+0000","last_active":"2023-10-07T07:57:50.735217+0000","last_peered":"2023-10-07T07:57:50.735217+0000","last_clean":"2023-10-07T07:57:50.735217+0000","last_became_active":"2023-10-07T07:57:50.729939+0000","last_became_peered":"2023-10-07T07:57:50.729939+0000","last_unstale":"2023-10-07T07:57:50.735217+0000","last_undegraded":"2023-10-07T07:57:50.735217+0000","last_fullsized":"2023-10-07T07:57:50.735217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:53:49.221151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735896+0000","last_change":"2023-10-07T07:57:50.730055+0000","last_active":"2023-10-07T07:57:50.735896+0000","last_peered":"2023-10-07T07:57:50.735896+0000","last_clean":"2023-10-07T07:57:50.735896+0000","last_became_active":"2023-10-07T07:57:50.729800+0000","last_became_peered":"2023-10-07T07:57:50.729800+0000","last_unstale":"2023-10-07T07:57:50.735896+0000","last_undegraded":"2023-10-07T07:57:50.735896+0000","last_fullsized":"2023-10-07T07:57:50.735896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:37:11.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725487+0000","last_change":"2023-10-07T07:57:50.718932+0000","last_active":"2023-10-07T07:57:50.725487+0000","last_peered":"2023-10-07T07:57:50.725487+0000","last_clean":"2023-10-07T07:57:50.725487+0000","last_became_active":"2023-10-07T07:57:50.718650+0000","last_became_peered":"2023-10-07T07:57:50.718650+0000","last_unstale":"2023-10-07T07:57:50.725487+0000","last_undegraded":"2023-10-07T07:57:50.725487+0000","last_fullsized":"2023-10-07T07:57:50.725487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:40.473036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.737108+0000","last_change":"2023-10-07T07:57:50.730279+0000","last_active":"2023-10-07T07:57:50.737108+0000","last_peered":"2023-10-07T07:57:50.737108+0000","last_clean":"2023-10-07T07:57:50.737108+0000","last_became_active":"2023-10-07T07:57:50.730004+0000","last_became_peered":"2023-10-07T07:57:50.730004+0000","last_unstale":"2023-10-07T07:57:50.737108+0000","last_undegraded":"2023-10-07T07:57:50.737108+0000","last_fullsized":"2023-10-07T07:57:50.737108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.728979+0000","last_change":"2023-10-07T07:57:50.720796+0000","last_active":"2023-10-07T07:57:50.728979+0000","last_peered":"2023-10-07T07:57:50.728979+0000","last_clean":"2023-10-07T07:57:50.728979+0000","last_became_active":"2023-10-07T07:57:50.720416+0000","last_became_peered":"2023-10-07T07:57:50.720416+0000","last_unstale":"2023-10-07T07:57:50.728979+0000","last_undegraded":"2023-10-07T07:57:50.728979+0000","last_fullsized":"2023-10-07T07:57:50.728979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:03:44.885267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743428+0000","last_change":"2023-10-07T07:57:50.741002+0000","last_active":"2023-10-07T07:57:50.743428+0000","last_peered":"2023-10-07T07:57:50.743428+0000","last_clean":"2023-10-07T07:57:50.743428+0000","last_became_active":"2023-10-07T07:57:50.740515+0000","last_became_peered":"2023-10-07T07:57:50.740515+0000","last_unstale":"2023-10-07T07:57:50.743428+0000","last_undegraded":"2023-10-07T07:57:50.743428+0000","last_fullsized":"2023-10-07T07:57:50.743428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:35:45.958299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.731000+0000","last_change":"2023-10-07T07:57:50.725365+0000","last_active":"2023-10-07T07:57:50.731000+0000","last_peered":"2023-10-07T07:57:50.731000+0000","last_clean":"2023-10-07T07:57:50.731000+0000","last_became_active":"2023-10-07T07:57:50.725076+0000","last_became_peered":"2023-10-07T07:57:50.725076+0000","last_unstale":"2023-10-07T07:57:50.731000+0000","last_undegraded":"2023-10-07T07:57:50.731000+0000","last_fullsized":"2023-10-07T07:57:50.731000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.736388+0000","last_change":"2023-10-07T07:57:50.722362+0000","last_active":"2023-10-07T07:57:50.736388+0000","last_peered":"2023-10-07T07:57:50.736388+0000","last_clean":"2023-10-07T07:57:50.736388+0000","last_became_active":"2023-10-07T07:57:50.722022+0000","last_became_peered":"2023-10-07T07:57:50.722022+0000","last_unstale":"2023-10-07T07:57:50.736388+0000","last_undegraded":"2023-10-07T07:57:50.736388+0000","last_fullsized":"2023-10-07T07:57:50.736388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:25:31.456924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742913+0000","last_change":"2023-10-07T07:57:49.726945+0000","last_active":"2023-10-07T07:57:50.742913+0000","last_peered":"2023-10-07T07:57:50.742913+0000","last_clean":"2023-10-07T07:57:50.742913+0000","last_became_active":"2023-10-07T07:57:49.726621+0000","last_became_peered":"2023-10-07T07:57:49.726621+0000","last_unstale":"2023-10-07T07:57:50.742913+0000","last_undegraded":"2023-10-07T07:57:50.742913+0000","last_fullsized":"2023-10-07T07:57:50.742913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741522+0000","last_change":"2023-10-07T07:57:50.740835+0000","last_active":"2023-10-07T07:57:50.741522+0000","last_peered":"2023-10-07T07:57:50.741522+0000","last_clean":"2023-10-07T07:57:50.741522+0000","last_became_active":"2023-10-07T07:57:50.740562+0000","last_became_peered":"2023-10-07T07:57:50.740562+0000","last_unstale":"2023-10-07T07:57:50.741522+0000","last_undegraded":"2023-10-07T07:57:50.741522+0000","last_fullsized":"2023-10-07T07:57:50.741522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:09:38.557090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738888+0000","last_change":"2023-10-07T07:57:49.725329+0000","last_active":"2023-10-07T07:57:50.738888+0000","last_peered":"2023-10-07T07:57:50.738888+0000","last_clean":"2023-10-07T07:57:50.738888+0000","last_became_active":"2023-10-07T07:57:49.724930+0000","last_became_peered":"2023-10-07T07:57:49.724930+0000","last_unstale":"2023-10-07T07:57:50.738888+0000","last_undegraded":"2023-10-07T07:57:50.738888+0000","last_fullsized":"2023-10-07T07:57:50.738888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725494+0000","last_change":"2023-10-07T07:57:50.724035+0000","last_active":"2023-10-07T07:57:50.725494+0000","last_peered":"2023-10-07T07:57:50.725494+0000","last_clean":"2023-10-07T07:57:50.725494+0000","last_became_active":"2023-10-07T07:57:50.723418+0000","last_became_peered":"2023-10-07T07:57:50.723418+0000","last_unstale":"2023-10-07T07:57:50.725494+0000","last_undegraded":"2023-10-07T07:57:50.725494+0000","last_fullsized":"2023-10-07T07:57:50.725494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741778+0000","last_change":"2023-10-07T07:57:49.727656+0000","last_active":"2023-10-07T07:57:50.741778+0000","last_peered":"2023-10-07T07:57:50.741778+0000","last_clean":"2023-10-07T07:57:50.741778+0000","last_became_active":"2023-10-07T07:57:49.727380+0000","last_became_peered":"2023-10-07T07:57:49.727380+0000","last_unstale":"2023-10-07T07:57:50.741778+0000","last_undegraded":"2023-10-07T07:57:50.741778+0000","last_fullsized":"2023-10-07T07:57:50.741778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:31.962724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732428+0000","last_change":"2023-10-07T07:57:50.727657+0000","last_active":"2023-10-07T07:57:50.732428+0000","last_peered":"2023-10-07T07:57:50.732428+0000","last_clean":"2023-10-07T07:57:50.732428+0000","last_became_active":"2023-10-07T07:57:50.727309+0000","last_became_peered":"2023-10-07T07:57:50.727309+0000","last_unstale":"2023-10-07T07:57:50.732428+0000","last_undegraded":"2023-10-07T07:57:50.732428+0000","last_fullsized":"2023-10-07T07:57:50.732428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743139+0000","last_change":"2023-10-07T07:57:49.729636+0000","last_active":"2023-10-07T07:57:50.743139+0000","last_peered":"2023-10-07T07:57:50.743139+0000","last_clean":"2023-10-07T07:57:50.743139+0000","last_became_active":"2023-10-07T07:57:49.725076+0000","last_became_peered":"2023-10-07T07:57:49.725076+0000","last_unstale":"2023-10-07T07:57:50.743139+0000","last_undegraded":"2023-10-07T07:57:50.743139+0000","last_fullsized":"2023-10-07T07:57:50.743139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:01.309236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743459+0000","last_change":"2023-10-07T07:57:50.742189+0000","last_active":"2023-10-07T07:57:50.743459+0000","last_peered":"2023-10-07T07:57:50.743459+0000","last_clean":"2023-10-07T07:57:50.743459+0000","last_became_active":"2023-10-07T07:57:50.741823+0000","last_became_peered":"2023-10-07T07:57:50.741823+0000","last_unstale":"2023-10-07T07:57:50.743459+0000","last_undegraded":"2023-10-07T07:57:50.743459+0000","last_fullsized":"2023-10-07T07:57:50.743459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:18:23.885325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738865+0000","last_change":"2023-10-07T07:57:49.724287+0000","last_active":"2023-10-07T07:57:50.738865+0000","last_peered":"2023-10-07T07:57:50.738865+0000","last_clean":"2023-10-07T07:57:50.738865+0000","last_became_active":"2023-10-07T07:57:49.723711+0000","last_became_peered":"2023-10-07T07:57:49.723711+0000","last_unstale":"2023-10-07T07:57:50.738865+0000","last_undegraded":"2023-10-07T07:57:50.738865+0000","last_fullsized":"2023-10-07T07:57:50.738865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726639+0000","last_change":"2023-10-07T07:57:50.723737+0000","last_active":"2023-10-07T07:57:50.726639+0000","last_peered":"2023-10-07T07:57:50.726639+0000","last_clean":"2023-10-07T07:57:50.726639+0000","last_became_active":"2023-10-07T07:57:50.723444+0000","last_became_peered":"2023-10-07T07:57:50.723444+0000","last_unstale":"2023-10-07T07:57:50.726639+0000","last_undegraded":"2023-10-07T07:57:50.726639+0000","last_fullsized":"2023-10-07T07:57:50.726639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.744159+0000","last_change":"2023-10-07T07:57:49.737749+0000","last_active":"2023-10-07T07:57:50.744159+0000","last_peered":"2023-10-07T07:57:50.744159+0000","last_clean":"2023-10-07T07:57:50.744159+0000","last_became_active":"2023-10-07T07:57:49.737477+0000","last_became_peered":"2023-10-07T07:57:49.737477+0000","last_unstale":"2023-10-07T07:57:50.744159+0000","last_undegraded":"2023-10-07T07:57:50.744159+0000","last_fullsized":"2023-10-07T07:57:50.744159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:10.280748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734802+0000","last_change":"2023-10-07T07:57:50.730239+0000","last_active":"2023-10-07T07:57:50.734802+0000","last_peered":"2023-10-07T07:57:50.734802+0000","last_clean":"2023-10-07T07:57:50.734802+0000","last_became_active":"2023-10-07T07:57:50.729972+0000","last_became_peered":"2023-10-07T07:57:50.729972+0000","last_unstale":"2023-10-07T07:57:50.734802+0000","last_undegraded":"2023-10-07T07:57:50.734802+0000","last_fullsized":"2023-10-07T07:57:50.734802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:03:06.838727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711739+0000","last_change":"2023-10-07T07:57:49.728260+0000","last_active":"2023-10-07T07:57:50.711739+0000","last_peered":"2023-10-07T07:57:50.711739+0000","last_clean":"2023-10-07T07:57:50.711739+0000","last_became_active":"2023-10-07T07:57:49.728019+0000","last_became_peered":"2023-10-07T07:57:49.728019+0000","last_unstale":"2023-10-07T07:57:50.711739+0000","last_undegraded":"2023-10-07T07:57:50.711739+0000","last_fullsized":"2023-10-07T07:57:50.711739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726987+0000","last_change":"2023-10-07T07:57:50.723700+0000","last_active":"2023-10-07T07:57:50.726987+0000","last_peered":"2023-10-07T07:57:50.726987+0000","last_clean":"2023-10-07T07:57:50.726987+0000","last_became_active":"2023-10-07T07:57:50.723420+0000","last_became_peered":"2023-10-07T07:57:50.723420+0000","last_unstale":"2023-10-07T07:57:50.726987+0000","last_undegraded":"2023-10-07T07:57:50.726987+0000","last_fullsized":"2023-10-07T07:57:50.726987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:26:00.336772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712589+0000","last_change":"2023-10-07T07:57:49.732565+0000","last_active":"2023-10-07T07:57:50.712589+0000","last_peered":"2023-10-07T07:57:50.712589+0000","last_clean":"2023-10-07T07:57:50.712589+0000","last_became_active":"2023-10-07T07:57:49.732250+0000","last_became_peered":"2023-10-07T07:57:49.732250+0000","last_unstale":"2023-10-07T07:57:50.712589+0000","last_undegraded":"2023-10-07T07:57:50.712589+0000","last_fullsized":"2023-10-07T07:57:50.712589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727015+0000","last_change":"2023-10-07T07:57:50.722227+0000","last_active":"2023-10-07T07:57:50.727015+0000","last_peered":"2023-10-07T07:57:50.727015+0000","last_clean":"2023-10-07T07:57:50.727015+0000","last_became_active":"2023-10-07T07:57:50.721953+0000","last_became_peered":"2023-10-07T07:57:50.721953+0000","last_unstale":"2023-10-07T07:57:50.727015+0000","last_undegraded":"2023-10-07T07:57:50.727015+0000","last_fullsized":"2023-10-07T07:57:50.727015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:50:56.892918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.709932+0000","last_change":"2023-10-07T07:57:49.720571+0000","last_active":"2023-10-07T07:57:50.709932+0000","last_peered":"2023-10-07T07:57:50.709932+0000","last_clean":"2023-10-07T07:57:50.709932+0000","last_became_active":"2023-10-07T07:57:49.720226+0000","last_became_peered":"2023-10-07T07:57:49.720226+0000","last_unstale":"2023-10-07T07:57:50.709932+0000","last_undegraded":"2023-10-07T07:57:50.709932+0000","last_fullsized":"2023-10-07T07:57:50.709932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740842+0000","last_change":"2023-10-07T07:57:50.724121+0000","last_active":"2023-10-07T07:57:50.740842+0000","last_peered":"2023-10-07T07:57:50.740842+0000","last_clean":"2023-10-07T07:57:50.740842+0000","last_became_active":"2023-10-07T07:57:50.723789+0000","last_became_peered":"2023-10-07T07:57:50.723789+0000","last_unstale":"2023-10-07T07:57:50.740842+0000","last_undegraded":"2023-10-07T07:57:50.740842+0000","last_fullsized":"2023-10-07T07:57:50.740842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.720435+0000","last_change":"2023-10-07T07:57:49.733414+0000","last_active":"2023-10-07T07:57:50.720435+0000","last_peered":"2023-10-07T07:57:50.720435+0000","last_clean":"2023-10-07T07:57:50.720435+0000","last_became_active":"2023-10-07T07:57:49.732736+0000","last_became_peered":"2023-10-07T07:57:49.732736+0000","last_unstale":"2023-10-07T07:57:50.720435+0000","last_undegraded":"2023-10-07T07:57:50.720435+0000","last_fullsized":"2023-10-07T07:57:50.720435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:49.692513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727348+0000","last_change":"2023-10-07T07:57:50.724743+0000","last_active":"2023-10-07T07:57:50.727348+0000","last_peered":"2023-10-07T07:57:50.727348+0000","last_clean":"2023-10-07T07:57:50.727348+0000","last_became_active":"2023-10-07T07:57:50.724501+0000","last_became_peered":"2023-10-07T07:57:50.724501+0000","last_unstale":"2023-10-07T07:57:50.727348+0000","last_undegraded":"2023-10-07T07:57:50.727348+0000","last_fullsized":"2023-10-07T07:57:50.727348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:03:23.294266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738823+0000","last_change":"2023-10-07T07:57:49.721843+0000","last_active":"2023-10-07T07:57:50.738823+0000","last_peered":"2023-10-07T07:57:50.738823+0000","last_clean":"2023-10-07T07:57:50.738823+0000","last_became_active":"2023-10-07T07:57:49.721533+0000","last_became_peered":"2023-10-07T07:57:49.721533+0000","last_unstale":"2023-10-07T07:57:50.738823+0000","last_undegraded":"2023-10-07T07:57:50.738823+0000","last_fullsized":"2023-10-07T07:57:50.738823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729050+0000","last_change":"2023-10-07T07:57:50.724096+0000","last_active":"2023-10-07T07:57:50.729050+0000","last_peered":"2023-10-07T07:57:50.729050+0000","last_clean":"2023-10-07T07:57:50.729050+0000","last_became_active":"2023-10-07T07:57:50.723782+0000","last_became_peered":"2023-10-07T07:57:50.723782+0000","last_unstale":"2023-10-07T07:57:50.729050+0000","last_undegraded":"2023-10-07T07:57:50.729050+0000","last_fullsized":"2023-10-07T07:57:50.729050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:59:47.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.746945+0000","last_change":"2023-10-07T07:57:49.720185+0000","last_active":"2023-10-07T07:57:50.746945+0000","last_peered":"2023-10-07T07:57:50.746945+0000","last_clean":"2023-10-07T07:57:50.746945+0000","last_became_active":"2023-10-07T07:57:49.719880+0000","last_became_peered":"2023-10-07T07:57:49.719880+0000","last_unstale":"2023-10-07T07:57:50.746945+0000","last_undegraded":"2023-10-07T07:57:50.746945+0000","last_fullsized":"2023-10-07T07:57:50.746945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:53:25.421958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732557+0000","last_change":"2023-10-07T07:57:50.726280+0000","last_active":"2023-10-07T07:57:50.732557+0000","last_peered":"2023-10-07T07:57:50.732557+0000","last_clean":"2023-10-07T07:57:50.732557+0000","last_became_active":"2023-10-07T07:57:50.725649+0000","last_became_peered":"2023-10-07T07:57:50.725649+0000","last_unstale":"2023-10-07T07:57:50.732557+0000","last_undegraded":"2023-10-07T07:57:50.732557+0000","last_fullsized":"2023-10-07T07:57:50.732557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:22:50.873696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739735+0000","last_change":"2023-10-07T07:57:49.732331+0000","last_active":"2023-10-07T07:57:50.739735+0000","last_peered":"2023-10-07T07:57:50.739735+0000","last_clean":"2023-10-07T07:57:50.739735+0000","last_became_active":"2023-10-07T07:57:49.732085+0000","last_became_peered":"2023-10-07T07:57:49.732085+0000","last_unstale":"2023-10-07T07:57:50.739735+0000","last_undegraded":"2023-10-07T07:57:50.739735+0000","last_fullsized":"2023-10-07T07:57:50.739735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727938+0000","last_change":"2023-10-07T07:57:50.724196+0000","last_active":"2023-10-07T07:57:50.727938+0000","last_peered":"2023-10-07T07:57:50.727938+0000","last_clean":"2023-10-07T07:57:50.727938+0000","last_became_active":"2023-10-07T07:57:50.723911+0000","last_became_peered":"2023-10-07T07:57:50.723911+0000","last_unstale":"2023-10-07T07:57:50.727938+0000","last_undegraded":"2023-10-07T07:57:50.727938+0000","last_fullsized":"2023-10-07T07:57:50.727938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:42.757604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712455+0000","last_change":"2023-10-07T07:57:49.734820+0000","last_active":"2023-10-07T07:57:50.712455+0000","last_peered":"2023-10-07T07:57:50.712455+0000","last_clean":"2023-10-07T07:57:50.712455+0000","last_became_active":"2023-10-07T07:57:49.732635+0000","last_became_peered":"2023-10-07T07:57:49.732635+0000","last_unstale":"2023-10-07T07:57:50.712455+0000","last_undegraded":"2023-10-07T07:57:50.712455+0000","last_fullsized":"2023-10-07T07:57:50.712455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:36:22.098994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743306+0000","last_change":"2023-10-07T07:57:50.740602+0000","last_active":"2023-10-07T07:57:50.743306+0000","last_peered":"2023-10-07T07:57:50.743306+0000","last_clean":"2023-10-07T07:57:50.743306+0000","last_became_active":"2023-10-07T07:57:50.740273+0000","last_became_peered":"2023-10-07T07:57:50.740273+0000","last_unstale":"2023-10-07T07:57:50.743306+0000","last_undegraded":"2023-10-07T07:57:50.743306+0000","last_fullsized":"2023-10-07T07:57:50.743306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:33:31.464165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.716494+0000","last_change":"2023-10-07T07:57:49.738688+0000","last_active":"2023-10-07T07:57:50.716494+0000","last_peered":"2023-10-07T07:57:50.716494+0000","last_clean":"2023-10-07T07:57:50.716494+0000","last_became_active":"2023-10-07T07:57:49.738387+0000","last_became_peered":"2023-10-07T07:57:49.738387+0000","last_unstale":"2023-10-07T07:57:50.716494+0000","last_undegraded":"2023-10-07T07:57:50.716494+0000","last_fullsized":"2023-10-07T07:57:50.716494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726565+0000","last_change":"2023-10-07T07:57:50.723301+0000","last_active":"2023-10-07T07:57:50.726565+0000","last_peered":"2023-10-07T07:57:50.726565+0000","last_clean":"2023-10-07T07:57:50.726565+0000","last_became_active":"2023-10-07T07:57:50.722920+0000","last_became_peered":"2023-10-07T07:57:50.722920+0000","last_unstale":"2023-10-07T07:57:50.726565+0000","last_undegraded":"2023-10-07T07:57:50.726565+0000","last_fullsized":"2023-10-07T07:57:50.726565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:32.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739604+0000","last_change":"2023-10-07T07:57:49.719344+0000","last_active":"2023-10-07T07:57:50.739604+0000","last_peered":"2023-10-07T07:57:50.739604+0000","last_clean":"2023-10-07T07:57:50.739604+0000","last_became_active":"2023-10-07T07:57:49.719053+0000","last_became_peered":"2023-10-07T07:57:49.719053+0000","last_unstale":"2023-10-07T07:57:50.739604+0000","last_undegraded":"2023-10-07T07:57:50.739604+0000","last_fullsized":"2023-10-07T07:57:50.739604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:32:15.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729136+0000","last_change":"2023-10-07T07:57:50.724500+0000","last_active":"2023-10-07T07:57:50.729136+0000","last_peered":"2023-10-07T07:57:50.729136+0000","last_clean":"2023-10-07T07:57:50.729136+0000","last_became_active":"2023-10-07T07:57:50.724175+0000","last_became_peered":"2023-10-07T07:57:50.724175+0000","last_unstale":"2023-10-07T07:57:50.729136+0000","last_undegraded":"2023-10-07T07:57:50.729136+0000","last_fullsized":"2023-10-07T07:57:50.729136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739659+0000","last_change":"2023-10-07T07:57:49.733130+0000","last_active":"2023-10-07T07:57:50.739659+0000","last_peered":"2023-10-07T07:57:50.739659+0000","last_clean":"2023-10-07T07:57:50.739659+0000","last_became_active":"2023-10-07T07:57:49.732810+0000","last_became_peered":"2023-10-07T07:57:49.732810+0000","last_unstale":"2023-10-07T07:57:50.739659+0000","last_undegraded":"2023-10-07T07:57:50.739659+0000","last_fullsized":"2023-10-07T07:57:50.739659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:32:15.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741983+0000","last_change":"2023-10-07T07:57:50.731887+0000","last_active":"2023-10-07T07:57:50.741983+0000","last_peered":"2023-10-07T07:57:50.741983+0000","last_clean":"2023-10-07T07:57:50.741983+0000","last_became_active":"2023-10-07T07:57:50.731535+0000","last_became_peered":"2023-10-07T07:57:50.731535+0000","last_unstale":"2023-10-07T07:57:50.741983+0000","last_undegraded":"2023-10-07T07:57:50.741983+0000","last_fullsized":"2023-10-07T07:57:50.741983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:40.473036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711292+0000","last_change":"2023-10-07T07:57:49.722522+0000","last_active":"2023-10-07T07:57:50.711292+0000","last_peered":"2023-10-07T07:57:50.711292+0000","last_clean":"2023-10-07T07:57:50.711292+0000","last_became_active":"2023-10-07T07:57:49.722225+0000","last_became_peered":"2023-10-07T07:57:49.722225+0000","last_unstale":"2023-10-07T07:57:50.711292+0000","last_undegraded":"2023-10-07T07:57:50.711292+0000","last_fullsized":"2023-10-07T07:57:50.711292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.719014+0000","last_change":"2023-10-07T07:57:50.717165+0000","last_active":"2023-10-07T07:57:50.719014+0000","last_peered":"2023-10-07T07:57:50.719014+0000","last_clean":"2023-10-07T07:57:50.719014+0000","last_became_active":"2023-10-07T07:57:50.716840+0000","last_became_peered":"2023-10-07T07:57:50.716840+0000","last_unstale":"2023-10-07T07:57:50.719014+0000","last_undegraded":"2023-10-07T07:57:50.719014+0000","last_fullsized":"2023-10-07T07:57:50.719014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:50.698077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712289+0000","last_change":"2023-10-07T07:57:49.735497+0000","last_active":"2023-10-07T07:57:50.712289+0000","last_peered":"2023-10-07T07:57:50.712289+0000","last_clean":"2023-10-07T07:57:50.712289+0000","last_became_active":"2023-10-07T07:57:49.735195+0000","last_became_peered":"2023-10-07T07:57:49.735195+0000","last_unstale":"2023-10-07T07:57:50.712289+0000","last_undegraded":"2023-10-07T07:57:50.712289+0000","last_fullsized":"2023-10-07T07:57:50.712289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:39.467472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742402+0000","last_change":"2023-10-07T07:57:50.741594+0000","last_active":"2023-10-07T07:57:50.742402+0000","last_peered":"2023-10-07T07:57:50.742402+0000","last_clean":"2023-10-07T07:57:50.742402+0000","last_became_active":"2023-10-07T07:57:50.741331+0000","last_became_peered":"2023-10-07T07:57:50.741331+0000","last_unstale":"2023-10-07T07:57:50.742402+0000","last_undegraded":"2023-10-07T07:57:50.742402+0000","last_fullsized":"2023-10-07T07:57:50.742402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:26:00.336772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733078+0000","last_change":"2023-10-07T07:57:50.726796+0000","last_active":"2023-10-07T07:57:50.733078+0000","last_peered":"2023-10-07T07:57:50.733078+0000","last_clean":"2023-10-07T07:57:50.733078+0000","last_became_active":"2023-10-07T07:57:50.725206+0000","last_became_peered":"2023-10-07T07:57:50.725206+0000","last_unstale":"2023-10-07T07:57:50.733078+0000","last_undegraded":"2023-10-07T07:57:50.733078+0000","last_fullsized":"2023-10-07T07:57:50.733078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:46:16.424520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.714634+0000","last_change":"2023-10-07T07:57:49.722494+0000","last_active":"2023-10-07T07:57:50.714634+0000","last_peered":"2023-10-07T07:57:50.714634+0000","last_clean":"2023-10-07T07:57:50.714634+0000","last_became_active":"2023-10-07T07:57:49.722237+0000","last_became_peered":"2023-10-07T07:57:49.722237+0000","last_unstale":"2023-10-07T07:57:50.714634+0000","last_undegraded":"2023-10-07T07:57:50.714634+0000","last_fullsized":"2023-10-07T07:57:50.714634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722017+0000","last_change":"2023-10-07T07:57:50.718217+0000","last_active":"2023-10-07T07:57:50.722017+0000","last_peered":"2023-10-07T07:57:50.722017+0000","last_clean":"2023-10-07T07:57:50.722017+0000","last_became_active":"2023-10-07T07:57:50.717791+0000","last_became_peered":"2023-10-07T07:57:50.717791+0000","last_unstale":"2023-10-07T07:57:50.722017+0000","last_undegraded":"2023-10-07T07:57:50.722017+0000","last_fullsized":"2023-10-07T07:57:50.722017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:42.757604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711986+0000","last_change":"2023-10-07T07:57:49.733338+0000","last_active":"2023-10-07T07:57:50.711986+0000","last_peered":"2023-10-07T07:57:50.711986+0000","last_clean":"2023-10-07T07:57:50.711986+0000","last_became_active":"2023-10-07T07:57:49.732478+0000","last_became_peered":"2023-10-07T07:57:49.732478+0000","last_unstale":"2023-10-07T07:57:50.711986+0000","last_undegraded":"2023-10-07T07:57:50.711986+0000","last_fullsized":"2023-10-07T07:57:50.711986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:01.309236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.709777+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T07:57:50.709777+0000","last_peered":"2023-10-07T07:57:50.709777+0000","last_clean":"2023-10-07T07:57:50.709777+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T07:57:50.709777+0000","last_undegraded":"2023-10-07T07:57:50.709777+0000","last_fullsized":"2023-10-07T07:57:50.709777+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727905+0000","last_change":"2023-10-07T07:57:50.723683+0000","last_active":"2023-10-07T07:57:50.727905+0000","last_peered":"2023-10-07T07:57:50.727905+0000","last_clean":"2023-10-07T07:57:50.727905+0000","last_became_active":"2023-10-07T07:57:50.723408+0000","last_became_peered":"2023-10-07T07:57:50.723408+0000","last_unstale":"2023-10-07T07:57:50.727905+0000","last_undegraded":"2023-10-07T07:57:50.727905+0000","last_fullsized":"2023-10-07T07:57:50.727905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:11.286312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712644+0000","last_change":"2023-10-07T07:57:49.733672+0000","last_active":"2023-10-07T07:57:50.712644+0000","last_peered":"2023-10-07T07:57:50.712644+0000","last_clean":"2023-10-07T07:57:50.712644+0000","last_became_active":"2023-10-07T07:57:49.733375+0000","last_became_peered":"2023-10-07T07:57:49.733375+0000","last_unstale":"2023-10-07T07:57:50.712644+0000","last_undegraded":"2023-10-07T07:57:50.712644+0000","last_fullsized":"2023-10-07T07:57:50.712644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.713200+0000","last_change":"2023-10-07T07:57:49.730927+0000","last_active":"2023-10-07T07:57:50.713200+0000","last_peered":"2023-10-07T07:57:50.713200+0000","last_clean":"2023-10-07T07:57:50.713200+0000","last_became_active":"2023-10-07T07:57:49.727008+0000","last_became_peered":"2023-10-07T07:57:49.727008+0000","last_unstale":"2023-10-07T07:57:50.713200+0000","last_undegraded":"2023-10-07T07:57:50.713200+0000","last_fullsized":"2023-10-07T07:57:50.713200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727827+0000","last_change":"2023-10-07T07:57:50.723292+0000","last_active":"2023-10-07T07:57:50.727827+0000","last_peered":"2023-10-07T07:57:50.727827+0000","last_clean":"2023-10-07T07:57:50.727827+0000","last_became_active":"2023-10-07T07:57:50.722388+0000","last_became_peered":"2023-10-07T07:57:50.722388+0000","last_unstale":"2023-10-07T07:57:50.727827+0000","last_undegraded":"2023-10-07T07:57:50.727827+0000","last_fullsized":"2023-10-07T07:57:50.727827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:53:57.122070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T07:57:53.552332+0000","last_change":"2023-10-07T07:57:49.736910+0000","last_active":"2023-10-07T07:57:53.552332+0000","last_peered":"2023-10-07T07:57:53.552332+0000","last_clean":"2023-10-07T07:57:53.552332+0000","last_became_active":"2023-10-07T07:57:49.736645+0000","last_became_peered":"2023-10-07T07:57:49.736645+0000","last_unstale":"2023-10-07T07:57:53.552332+0000","last_undegraded":"2023-10-07T07:57:53.552332+0000","last_fullsized":"2023-10-07T07:57:53.552332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726999+0000","last_change":"2023-10-07T07:57:50.721660+0000","last_active":"2023-10-07T07:57:50.726999+0000","last_peered":"2023-10-07T07:57:50.726999+0000","last_clean":"2023-10-07T07:57:50.726999+0000","last_became_active":"2023-10-07T07:57:50.721390+0000","last_became_peered":"2023-10-07T07:57:50.721390+0000","last_unstale":"2023-10-07T07:57:50.726999+0000","last_undegraded":"2023-10-07T07:57:50.726999+0000","last_fullsized":"2023-10-07T07:57:50.726999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:11.286312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.745865+0000","last_change":"2023-10-07T07:57:49.737869+0000","last_active":"2023-10-07T07:57:50.745865+0000","last_peered":"2023-10-07T07:57:50.745865+0000","last_clean":"2023-10-07T07:57:50.745865+0000","last_became_active":"2023-10-07T07:57:49.737571+0000","last_became_peered":"2023-10-07T07:57:49.737571+0000","last_unstale":"2023-10-07T07:57:50.745865+0000","last_undegraded":"2023-10-07T07:57:50.745865+0000","last_fullsized":"2023-10-07T07:57:50.745865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:49.692513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722039+0000","last_change":"2023-10-07T07:57:50.718352+0000","last_active":"2023-10-07T07:57:50.722039+0000","last_peered":"2023-10-07T07:57:50.722039+0000","last_clean":"2023-10-07T07:57:50.722039+0000","last_became_active":"2023-10-07T07:57:50.718091+0000","last_became_peered":"2023-10-07T07:57:50.718091+0000","last_unstale":"2023-10-07T07:57:50.722039+0000","last_undegraded":"2023-10-07T07:57:50.722039+0000","last_fullsized":"2023-10-07T07:57:50.722039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:37:11.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.714964+0000","last_change":"2023-10-07T07:57:49.730601+0000","last_active":"2023-10-07T07:57:50.714964+0000","last_peered":"2023-10-07T07:57:50.714964+0000","last_clean":"2023-10-07T07:57:50.714964+0000","last_became_active":"2023-10-07T07:57:49.730370+0000","last_became_peered":"2023-10-07T07:57:49.730370+0000","last_unstale":"2023-10-07T07:57:50.714964+0000","last_undegraded":"2023-10-07T07:57:50.714964+0000","last_fullsized":"2023-10-07T07:57:50.714964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729113+0000","last_change":"2023-10-07T07:57:50.723162+0000","last_active":"2023-10-07T07:57:50.729113+0000","last_peered":"2023-10-07T07:57:50.729113+0000","last_clean":"2023-10-07T07:57:50.729113+0000","last_became_active":"2023-10-07T07:57:50.722848+0000","last_became_peered":"2023-10-07T07:57:50.722848+0000","last_unstale":"2023-10-07T07:57:50.729113+0000","last_undegraded":"2023-10-07T07:57:50.729113+0000","last_fullsized":"2023-10-07T07:57:50.729113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712342+0000","last_change":"2023-10-07T07:57:49.731859+0000","last_active":"2023-10-07T07:57:50.712342+0000","last_peered":"2023-10-07T07:57:50.712342+0000","last_clean":"2023-10-07T07:57:50.712342+0000","last_became_active":"2023-10-07T07:57:49.731613+0000","last_became_peered":"2023-10-07T07:57:49.731613+0000","last_unstale":"2023-10-07T07:57:50.712342+0000","last_undegraded":"2023-10-07T07:57:50.712342+0000","last_fullsized":"2023-10-07T07:57:50.712342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727214+0000","last_change":"2023-10-07T07:57:50.723167+0000","last_active":"2023-10-07T07:57:50.727214+0000","last_peered":"2023-10-07T07:57:50.727214+0000","last_clean":"2023-10-07T07:57:50.727214+0000","last_became_active":"2023-10-07T07:57:50.722863+0000","last_became_peered":"2023-10-07T07:57:50.722863+0000","last_unstale":"2023-10-07T07:57:50.727214+0000","last_undegraded":"2023-10-07T07:57:50.727214+0000","last_fullsized":"2023-10-07T07:57:50.727214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711876+0000","last_change":"2023-10-07T07:57:49.725087+0000","last_active":"2023-10-07T07:57:50.711876+0000","last_peered":"2023-10-07T07:57:50.711876+0000","last_clean":"2023-10-07T07:57:50.711876+0000","last_became_active":"2023-10-07T07:57:49.724027+0000","last_became_peered":"2023-10-07T07:57:49.724027+0000","last_unstale":"2023-10-07T07:57:50.711876+0000","last_undegraded":"2023-10-07T07:57:50.711876+0000","last_fullsized":"2023-10-07T07:57:50.711876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722299+0000","last_change":"2023-10-07T07:57:50.717348+0000","last_active":"2023-10-07T07:57:50.722299+0000","last_peered":"2023-10-07T07:57:50.722299+0000","last_clean":"2023-10-07T07:57:50.722299+0000","last_became_active":"2023-10-07T07:57:50.716959+0000","last_became_peered":"2023-10-07T07:57:50.716959+0000","last_unstale":"2023-10-07T07:57:50.722299+0000","last_undegraded":"2023-10-07T07:57:50.722299+0000","last_fullsized":"2023-10-07T07:57:50.722299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742144+0000","last_change":"2023-10-07T07:57:49.732599+0000","last_active":"2023-10-07T07:57:50.742144+0000","last_peered":"2023-10-07T07:57:50.742144+0000","last_clean":"2023-10-07T07:57:50.742144+0000","last_became_active":"2023-10-07T07:57:49.732088+0000","last_became_peered":"2023-10-07T07:57:49.732088+0000","last_unstale":"2023-10-07T07:57:50.742144+0000","last_undegraded":"2023-10-07T07:57:50.742144+0000","last_fullsized":"2023-10-07T07:57:50.742144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:31.103335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729208+0000","last_change":"2023-10-07T07:57:50.725007+0000","last_active":"2023-10-07T07:57:50.729208+0000","last_peered":"2023-10-07T07:57:50.729208+0000","last_clean":"2023-10-07T07:57:50.729208+0000","last_became_active":"2023-10-07T07:57:50.723895+0000","last_became_peered":"2023-10-07T07:57:50.723895+0000","last_unstale":"2023-10-07T07:57:50.729208+0000","last_undegraded":"2023-10-07T07:57:50.729208+0000","last_fullsized":"2023-10-07T07:57:50.729208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:13:37.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712397+0000","last_change":"2023-10-07T07:57:49.725916+0000","last_active":"2023-10-07T07:57:50.712397+0000","last_peered":"2023-10-07T07:57:50.712397+0000","last_clean":"2023-10-07T07:57:50.712397+0000","last_became_active":"2023-10-07T07:57:49.725648+0000","last_became_peered":"2023-10-07T07:57:49.725648+0000","last_unstale":"2023-10-07T07:57:50.712397+0000","last_undegraded":"2023-10-07T07:57:50.712397+0000","last_fullsized":"2023-10-07T07:57:50.712397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.736361+0000","last_change":"2023-10-07T07:57:50.720453+0000","last_active":"2023-10-07T07:57:50.736361+0000","last_peered":"2023-10-07T07:57:50.736361+0000","last_clean":"2023-10-07T07:57:50.736361+0000","last_became_active":"2023-10-07T07:57:50.719863+0000","last_became_peered":"2023-10-07T07:57:50.719863+0000","last_unstale":"2023-10-07T07:57:50.736361+0000","last_undegraded":"2023-10-07T07:57:50.736361+0000","last_fullsized":"2023-10-07T07:57:50.736361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:21:30.452139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739318+0000","last_change":"2023-10-07T07:57:49.729606+0000","last_active":"2023-10-07T07:57:50.739318+0000","last_peered":"2023-10-07T07:57:50.739318+0000","last_clean":"2023-10-07T07:57:50.739318+0000","last_became_active":"2023-10-07T07:57:49.729372+0000","last_became_peered":"2023-10-07T07:57:49.729372+0000","last_unstale":"2023-10-07T07:57:50.739318+0000","last_undegraded":"2023-10-07T07:57:50.739318+0000","last_fullsized":"2023-10-07T07:57:50.739318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735545+0000","last_change":"2023-10-07T07:57:50.734705+0000","last_active":"2023-10-07T07:57:50.735545+0000","last_peered":"2023-10-07T07:57:50.735545+0000","last_clean":"2023-10-07T07:57:50.735545+0000","last_became_active":"2023-10-07T07:57:50.733706+0000","last_became_peered":"2023-10-07T07:57:50.733706+0000","last_unstale":"2023-10-07T07:57:50.735545+0000","last_undegraded":"2023-10-07T07:57:50.735545+0000","last_fullsized":"2023-10-07T07:57:50.735545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:56:34.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739193+0000","last_change":"2023-10-07T07:57:49.724649+0000","last_active":"2023-10-07T07:57:50.739193+0000","last_peered":"2023-10-07T07:57:50.739193+0000","last_clean":"2023-10-07T07:57:50.739193+0000","last_became_active":"2023-10-07T07:57:49.724368+0000","last_became_peered":"2023-10-07T07:57:49.724368+0000","last_unstale":"2023-10-07T07:57:50.739193+0000","last_undegraded":"2023-10-07T07:57:50.739193+0000","last_fullsized":"2023-10-07T07:57:50.739193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733770+0000","last_change":"2023-10-07T07:57:50.730366+0000","last_active":"2023-10-07T07:57:50.733770+0000","last_peered":"2023-10-07T07:57:50.733770+0000","last_clean":"2023-10-07T07:57:50.733770+0000","last_became_active":"2023-10-07T07:57:50.730012+0000","last_became_peered":"2023-10-07T07:57:50.730012+0000","last_unstale":"2023-10-07T07:57:50.733770+0000","last_undegraded":"2023-10-07T07:57:50.733770+0000","last_fullsized":"2023-10-07T07:57:50.733770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:15:16.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740610+0000","last_change":"2023-10-07T07:57:50.721978+0000","last_active":"2023-10-07T07:57:50.740610+0000","last_peered":"2023-10-07T07:57:50.740610+0000","last_clean":"2023-10-07T07:57:50.740610+0000","last_became_active":"2023-10-07T07:57:50.719507+0000","last_became_peered":"2023-10-07T07:57:50.719507+0000","last_unstale":"2023-10-07T07:57:50.740610+0000","last_undegraded":"2023-10-07T07:57:50.740610+0000","last_fullsized":"2023-10-07T07:57:50.740610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:32.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.730933+0000","last_change":"2023-10-07T07:57:50.724692+0000","last_active":"2023-10-07T07:57:50.730933+0000","last_peered":"2023-10-07T07:57:50.730933+0000","last_clean":"2023-10-07T07:57:50.730933+0000","last_became_active":"2023-10-07T07:57:50.724411+0000","last_became_peered":"2023-10-07T07:57:50.724411+0000","last_unstale":"2023-10-07T07:57:50.730933+0000","last_undegraded":"2023-10-07T07:57:50.730933+0000","last_fullsized":"2023-10-07T07:57:50.730933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732488+0000","last_change":"2023-10-07T07:57:50.727607+0000","last_active":"2023-10-07T07:57:50.732488+0000","last_peered":"2023-10-07T07:57:50.732488+0000","last_clean":"2023-10-07T07:57:50.732488+0000","last_became_active":"2023-10-07T07:57:50.724048+0000","last_became_peered":"2023-10-07T07:57:50.724048+0000","last_unstale":"2023-10-07T07:57:50.732488+0000","last_undegraded":"2023-10-07T07:57:50.732488+0000","last_fullsized":"2023-10-07T07:57:50.732488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:38:14.739161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734147+0000","last_change":"2023-10-07T07:57:50.726261+0000","last_active":"2023-10-07T07:57:50.734147+0000","last_peered":"2023-10-07T07:57:50.734147+0000","last_clean":"2023-10-07T07:57:50.734147+0000","last_became_active":"2023-10-07T07:57:50.725636+0000","last_became_peered":"2023-10-07T07:57:50.725636+0000","last_unstale":"2023-10-07T07:57:50.734147+0000","last_undegraded":"2023-10-07T07:57:50.734147+0000","last_fullsized":"2023-10-07T07:57:50.734147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:56:34.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738656+0000","last_change":"2023-10-07T07:57:50.735497+0000","last_active":"2023-10-07T07:57:50.738656+0000","last_peered":"2023-10-07T07:57:50.738656+0000","last_clean":"2023-10-07T07:57:50.738656+0000","last_became_active":"2023-10-07T07:57:50.735172+0000","last_became_peered":"2023-10-07T07:57:50.735172+0000","last_unstale":"2023-10-07T07:57:50.738656+0000","last_undegraded":"2023-10-07T07:57:50.738656+0000","last_fullsized":"2023-10-07T07:57:50.738656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725429+0000","last_change":"2023-10-07T07:57:50.718981+0000","last_active":"2023-10-07T07:57:50.725429+0000","last_peered":"2023-10-07T07:57:50.725429+0000","last_clean":"2023-10-07T07:57:50.725429+0000","last_became_active":"2023-10-07T07:57:50.718699+0000","last_became_peered":"2023-10-07T07:57:50.718699+0000","last_unstale":"2023-10-07T07:57:50.725429+0000","last_undegraded":"2023-10-07T07:57:50.725429+0000","last_fullsized":"2023-10-07T07:57:50.725429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:36:23.104558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740872+0000","last_change":"2023-10-07T07:57:50.724708+0000","last_active":"2023-10-07T07:57:50.740872+0000","last_peered":"2023-10-07T07:57:50.740872+0000","last_clean":"2023-10-07T07:57:50.740872+0000","last_became_active":"2023-10-07T07:57:50.724343+0000","last_became_peered":"2023-10-07T07:57:50.724343+0000","last_unstale":"2023-10-07T07:57:50.740872+0000","last_undegraded":"2023-10-07T07:57:50.740872+0000","last_fullsized":"2023-10-07T07:57:50.740872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:38:53.980034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725505+0000","last_change":"2023-10-07T07:57:50.724233+0000","last_active":"2023-10-07T07:57:50.725505+0000","last_peered":"2023-10-07T07:57:50.725505+0000","last_clean":"2023-10-07T07:57:50.725505+0000","last_became_active":"2023-10-07T07:57:50.723770+0000","last_became_peered":"2023-10-07T07:57:50.723770+0000","last_unstale":"2023-10-07T07:57:50.725505+0000","last_undegraded":"2023-10-07T07:57:50.725505+0000","last_fullsized":"2023-10-07T07:57:50.725505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:22:50.873696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729190+0000","last_change":"2023-10-07T07:57:50.724839+0000","last_active":"2023-10-07T07:57:50.729190+0000","last_peered":"2023-10-07T07:57:50.729190+0000","last_clean":"2023-10-07T07:57:50.729190+0000","last_became_active":"2023-10-07T07:57:50.724489+0000","last_became_peered":"2023-10-07T07:57:50.724489+0000","last_unstale":"2023-10-07T07:57:50.729190+0000","last_undegraded":"2023-10-07T07:57:50.729190+0000","last_fullsized":"2023-10-07T07:57:50.729190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:38:11.263837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741272+0000","last_change":"2023-10-07T07:57:50.740752+0000","last_active":"2023-10-07T07:57:50.741272+0000","last_peered":"2023-10-07T07:57:50.741272+0000","last_clean":"2023-10-07T07:57:50.741272+0000","last_became_active":"2023-10-07T07:57:50.740464+0000","last_became_peered":"2023-10-07T07:57:50.740464+0000","last_unstale":"2023-10-07T07:57:50.741272+0000","last_undegraded":"2023-10-07T07:57:50.741272+0000","last_fullsized":"2023-10-07T07:57:50.741272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733824+0000","last_change":"2023-10-07T07:57:50.729375+0000","last_active":"2023-10-07T07:57:50.733824+0000","last_peered":"2023-10-07T07:57:50.733824+0000","last_clean":"2023-10-07T07:57:50.733824+0000","last_became_active":"2023-10-07T07:57:50.729038+0000","last_became_peered":"2023-10-07T07:57:50.729038+0000","last_unstale":"2023-10-07T07:57:50.733824+0000","last_undegraded":"2023-10-07T07:57:50.733824+0000","last_fullsized":"2023-10-07T07:57:50.733824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724924+0000","last_change":"2023-10-07T07:57:50.718717+0000","last_active":"2023-10-07T07:57:50.724924+0000","last_peered":"2023-10-07T07:57:50.724924+0000","last_clean":"2023-10-07T07:57:50.724924+0000","last_became_active":"2023-10-07T07:57:50.718457+0000","last_became_peered":"2023-10-07T07:57:50.718457+0000","last_unstale":"2023-10-07T07:57:50.724924+0000","last_undegraded":"2023-10-07T07:57:50.724924+0000","last_fullsized":"2023-10-07T07:57:50.724924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:53:57.122070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735843+0000","last_change":"2023-10-07T07:57:50.734975+0000","last_active":"2023-10-07T07:57:50.735843+0000","last_peered":"2023-10-07T07:57:50.735843+0000","last_clean":"2023-10-07T07:57:50.735843+0000","last_became_active":"2023-10-07T07:57:50.734441+0000","last_became_peered":"2023-10-07T07:57:50.734441+0000","last_unstale":"2023-10-07T07:57:50.735843+0000","last_undegraded":"2023-10-07T07:57:50.735843+0000","last_fullsized":"2023-10-07T07:57:50.735843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":665751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":75867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":77493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298088,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073752,"statfs":{"total":96636764160,"available":96331522048,"internally_reserved":0,"allocated":819200,"data_stored":665773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T07:57:59.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T07:57:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.976+0000 7f42fff0c700 1 Processor -- start 2023-10-07T07:57:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.977+0000 7f42fff0c700 1 -- start start 2023-10-07T07:57:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.977+0000 7f42fff0c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42f81570a0 0x7f42f80c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.977+0000 7f42fff0c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42f8157470 0x7f42f806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.977+0000 7f42fff0c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f80c3b80 con 0x7f42f806bb60 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.977+0000 7f42fff0c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f806bfe0 con 0x7f42f81570a0 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.977+0000 7f42fff0c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f80c2f70 con 0x7f42f8157470 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fdca8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42f81570a0 0x7f42f80c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fdca8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42f81570a0 0x7f42f80c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47876/0 (socket says 172.21.15.7:47876) 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fdca8700 1 -- 172.21.15.7:0/637289617 learned_addr learned my addr 172.21.15.7:0/637289617 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fd4a7700 1 --2- 172.21.15.7:0/637289617 >> v2:172.21.15.172:3300/0 conn(0x7f42f8157470 0x7f42f806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fdca8700 1 -- 172.21.15.7:0/637289617 >> v2:172.21.15.172:3300/0 conn(0x7f42f8157470 msgr2=0x7f42f806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fdca8700 1 --2- 172.21.15.7:0/637289617 >> v2:172.21.15.172:3300/0 conn(0x7f42f8157470 0x7f42f806b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fdca8700 1 -- 172.21.15.7:0/637289617 >> v1:172.21.15.172:6789/0 conn(0x7f42f806bb60 legacy=0x7f42f806cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:57:59.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fdca8700 1 -- 172.21.15.7:0/637289617 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80c3250 con 0x7f42f81570a0 2023-10-07T07:57:59.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fdca8700 1 --2- 172.21.15.7:0/637289617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f81570a0 0x7f42f80c8800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f42f806df20 tx=0x7f42f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2707753499226a02 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:59.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fcca6700 1 -- 172.21.15.7:0/637289617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f400b820 con 0x7f42f81570a0 2023-10-07T07:57:59.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fcca6700 1 -- 172.21.15.7:0/637289617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f400b980 con 0x7f42f81570a0 2023-10-07T07:57:59.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.978+0000 7f42fcca6700 1 -- 172.21.15.7:0/637289617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f400bd20 con 0x7f42f81570a0 2023-10-07T07:57:59.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.979+0000 7f42fff0c700 1 -- 172.21.15.7:0/637289617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f81570a0 msgr2=0x7f42f80c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.979+0000 7f42fff0c700 1 --2- 172.21.15.7:0/637289617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f81570a0 0x7f42f80c8800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f42f806df20 tx=0x7f42f4009250 comp rx=0 tx=0).stop 2023-10-07T07:57:59.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.979+0000 7f42fff0c700 1 -- 172.21.15.7:0/637289617 shutdown_connections 2023-10-07T07:57:59.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.979+0000 7f42fff0c700 1 --2- 172.21.15.7:0/637289617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f81570a0 0x7f42f80c8800 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.979+0000 7f42fff0c700 1 --2- 172.21.15.7:0/637289617 >> v2:172.21.15.172:3300/0 conn(0x7f42f8157470 0x7f42f806b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.979+0000 7f42fff0c700 1 -- 172.21.15.7:0/637289617 >> 172.21.15.7:0/637289617 conn(0x7f42f8067ac0 msgr2=0x7f42f80ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:57:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.979+0000 7f42fff0c700 1 -- 172.21.15.7:0/637289617 shutdown_connections 2023-10-07T07:57:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.979+0000 7f42fff0c700 1 -- 172.21.15.7:0/637289617 wait complete. 2023-10-07T07:57:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.980+0000 7f42fff0c700 1 Processor -- start 2023-10-07T07:57:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.980+0000 7f42fff0c700 1 -- start start 2023-10-07T07:57:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fff0c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42f806bb60 0x7f42f816ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fff0c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f81570a0 0x7f42f816a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fff0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8157470 0x7f42f8068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fff0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f8068640 con 0x7f42f8157470 2023-10-07T07:57:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fff0c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f813a3e0 con 0x7f42f806bb60 2023-10-07T07:57:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fff0c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f8069110 con 0x7f42f81570a0 2023-10-07T07:57:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fe4a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8157470 0x7f42f8068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fe4a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8157470 0x7f42f8068100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47884/0 (socket says 172.21.15.7:47884) 2023-10-07T07:57:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fe4a9700 1 -- 172.21.15.7:0/20175912 learned_addr learned my addr 172.21.15.7:0/20175912 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:57:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fdca8700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42f806bb60 0x7f42f816ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fd4a7700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f81570a0 0x7f42f816a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fe4a9700 1 -- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f81570a0 msgr2=0x7f42f816a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fe4a9700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f81570a0 0x7f42f816a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fe4a9700 1 -- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42f806bb60 msgr2=0x7f42f816ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:57:59.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fe4a9700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42f806bb60 0x7f42f816ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:57:59.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.981+0000 7f42fe4a9700 1 -- 172.21.15.7:0/20175912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f40094f0 con 0x7f42f8157470 2023-10-07T07:57:59.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.982+0000 7f42fe4a9700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8157470 0x7f42f8068100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f42f000bc30 tx=0x7f42f0009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:59.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.982+0000 7f42edffb700 1 -- 172.21.15.7:0/20175912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f000c7e0 con 0x7f42f8157470 2023-10-07T07:57:59.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.982+0000 7f42edffb700 1 -- 172.21.15.7:0/20175912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f000c980 con 0x7f42f8157470 2023-10-07T07:57:59.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.982+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f8068870 con 0x7f42f8157470 2023-10-07T07:57:59.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.982+0000 7f42edffb700 1 -- 172.21.15.7:0/20175912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f000cd40 con 0x7f42f8157470 2023-10-07T07:57:59.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.982+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f8068e30 con 0x7f42f8157470 2023-10-07T07:57:59.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.983+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f80716c0 con 0x7f42f8157470 2023-10-07T07:57:59.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.983+0000 7f42edffb700 1 -- 172.21.15.7:0/20175912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42f0014620 con 0x7f42f8157470 2023-10-07T07:57:59.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.984+0000 7f42edffb700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f42dc066f30 0x7f42dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:57:59.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.984+0000 7f42edffb700 1 -- 172.21.15.7:0/20175912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f42f0084400 con 0x7f42f8157470 2023-10-07T07:57:59.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.984+0000 7f42fdca8700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f42dc066f30 0x7f42dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:57:59.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.984+0000 7f42fdca8700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f42dc066f30 0x7f42dc0693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f42f806df20 tx=0x7f42f4016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:57:59.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:57:59.988+0000 7f42edffb700 1 -- 172.21.15.7:0/20175912 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42f004c5a0 con 0x7f42f8157470 2023-10-07T07:58:00.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.187+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42f813a640 con 0x7f42dc066f30 2023-10-07T07:58:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.194+0000 7f42edffb700 1 -- 172.21.15.7:0/20175912 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264512 (secure 0 0 0) 0x7f42f813a640 con 0x7f42dc066f30 2023-10-07T07:58:00.198 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:00.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f42dc066f30 msgr2=0x7f42dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f42dc066f30 0x7f42dc0693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f42f806df20 tx=0x7f42f4016000 comp rx=0 tx=0).stop 2023-10-07T07:58:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8157470 msgr2=0x7f42f8068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8157470 0x7f42f8068100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f42f000bc30 tx=0x7f42f0009820 comp rx=0 tx=0).stop 2023-10-07T07:58:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 shutdown_connections 2023-10-07T07:58:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f42dc066f30 0x7f42dc0693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8157470 0x7f42f8068100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42f806bb60 0x7f42f816ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 --2- 172.21.15.7:0/20175912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f81570a0 0x7f42f816a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 >> 172.21.15.7:0/20175912 conn(0x7f42f8067ac0 msgr2=0x7f42f80ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 shutdown_connections 2023-10-07T07:58:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.198+0000 7f42fff0c700 1 -- 172.21.15.7:0/20175912 wait complete. 2023-10-07T07:58:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T07:58:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-07T07:58:00.027587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381224,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593496,"statfs":{"total":773094113280,"available":770655739904,"internally_reserved":0,"allocated":2990080,"data_stored":1787852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.321691"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733747+0000","last_change":"2023-10-07T07:57:50.730298+0000","last_active":"2023-10-07T07:57:50.733747+0000","last_peered":"2023-10-07T07:57:50.733747+0000","last_clean":"2023-10-07T07:57:50.733747+0000","last_became_active":"2023-10-07T07:57:50.729981+0000","last_became_peered":"2023-10-07T07:57:50.729981+0000","last_unstale":"2023-10-07T07:57:50.733747+0000","last_undegraded":"2023-10-07T07:57:50.733747+0000","last_fullsized":"2023-10-07T07:57:50.733747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:13:37.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734352+0000","last_change":"2023-10-07T07:57:50.726555+0000","last_active":"2023-10-07T07:57:50.734352+0000","last_peered":"2023-10-07T07:57:50.734352+0000","last_clean":"2023-10-07T07:57:50.734352+0000","last_became_active":"2023-10-07T07:57:50.726036+0000","last_became_peered":"2023-10-07T07:57:50.726036+0000","last_unstale":"2023-10-07T07:57:50.734352+0000","last_undegraded":"2023-10-07T07:57:50.734352+0000","last_fullsized":"2023-10-07T07:57:50.734352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:03:06.838727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726644+0000","last_change":"2023-10-07T07:57:50.722937+0000","last_active":"2023-10-07T07:57:50.726644+0000","last_peered":"2023-10-07T07:57:50.726644+0000","last_clean":"2023-10-07T07:57:50.726644+0000","last_became_active":"2023-10-07T07:57:50.722662+0000","last_became_peered":"2023-10-07T07:57:50.722662+0000","last_unstale":"2023-10-07T07:57:50.726644+0000","last_undegraded":"2023-10-07T07:57:50.726644+0000","last_fullsized":"2023-10-07T07:57:50.726644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:15:16.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741568+0000","last_change":"2023-10-07T07:57:50.725792+0000","last_active":"2023-10-07T07:57:50.741568+0000","last_peered":"2023-10-07T07:57:50.741568+0000","last_clean":"2023-10-07T07:57:50.741568+0000","last_became_active":"2023-10-07T07:57:50.725545+0000","last_became_peered":"2023-10-07T07:57:50.725545+0000","last_unstale":"2023-10-07T07:57:50.741568+0000","last_undegraded":"2023-10-07T07:57:50.741568+0000","last_fullsized":"2023-10-07T07:57:50.741568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739501+0000","last_change":"2023-10-07T07:57:50.737070+0000","last_active":"2023-10-07T07:57:50.739501+0000","last_peered":"2023-10-07T07:57:50.739501+0000","last_clean":"2023-10-07T07:57:50.739501+0000","last_became_active":"2023-10-07T07:57:50.734411+0000","last_became_peered":"2023-10-07T07:57:50.734411+0000","last_unstale":"2023-10-07T07:57:50.739501+0000","last_undegraded":"2023-10-07T07:57:50.739501+0000","last_fullsized":"2023-10-07T07:57:50.739501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:03:23.294266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740242+0000","last_change":"2023-10-07T07:57:50.719875+0000","last_active":"2023-10-07T07:57:50.740242+0000","last_peered":"2023-10-07T07:57:50.740242+0000","last_clean":"2023-10-07T07:57:50.740242+0000","last_became_active":"2023-10-07T07:57:50.719322+0000","last_became_peered":"2023-10-07T07:57:50.719322+0000","last_unstale":"2023-10-07T07:57:50.740242+0000","last_undegraded":"2023-10-07T07:57:50.740242+0000","last_fullsized":"2023-10-07T07:57:50.740242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732962+0000","last_change":"2023-10-07T07:57:50.728894+0000","last_active":"2023-10-07T07:57:50.732962+0000","last_peered":"2023-10-07T07:57:50.732962+0000","last_clean":"2023-10-07T07:57:50.732962+0000","last_became_active":"2023-10-07T07:57:50.726313+0000","last_became_peered":"2023-10-07T07:57:50.726313+0000","last_unstale":"2023-10-07T07:57:50.732962+0000","last_undegraded":"2023-10-07T07:57:50.732962+0000","last_fullsized":"2023-10-07T07:57:50.732962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.736796+0000","last_change":"2023-10-07T07:57:50.730368+0000","last_active":"2023-10-07T07:57:50.736796+0000","last_peered":"2023-10-07T07:57:50.736796+0000","last_clean":"2023-10-07T07:57:50.736796+0000","last_became_active":"2023-10-07T07:57:50.730094+0000","last_became_peered":"2023-10-07T07:57:50.730094+0000","last_unstale":"2023-10-07T07:57:50.736796+0000","last_undegraded":"2023-10-07T07:57:50.736796+0000","last_fullsized":"2023-10-07T07:57:50.736796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:53:49.221151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734017+0000","last_change":"2023-10-07T07:57:50.726948+0000","last_active":"2023-10-07T07:57:50.734017+0000","last_peered":"2023-10-07T07:57:50.734017+0000","last_clean":"2023-10-07T07:57:50.734017+0000","last_became_active":"2023-10-07T07:57:50.726622+0000","last_became_peered":"2023-10-07T07:57:50.726622+0000","last_unstale":"2023-10-07T07:57:50.734017+0000","last_undegraded":"2023-10-07T07:57:50.734017+0000","last_fullsized":"2023-10-07T07:57:50.734017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:38:53.980034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.745995+0000","last_change":"2023-10-07T07:57:50.745365+0000","last_active":"2023-10-07T07:57:50.745995+0000","last_peered":"2023-10-07T07:57:50.745995+0000","last_clean":"2023-10-07T07:57:50.745995+0000","last_became_active":"2023-10-07T07:57:50.745061+0000","last_became_peered":"2023-10-07T07:57:50.745061+0000","last_unstale":"2023-10-07T07:57:50.745995+0000","last_undegraded":"2023-10-07T07:57:50.745995+0000","last_fullsized":"2023-10-07T07:57:50.745995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:09:38.557090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733023+0000","last_change":"2023-10-07T07:57:50.729317+0000","last_active":"2023-10-07T07:57:50.733023+0000","last_peered":"2023-10-07T07:57:50.733023+0000","last_clean":"2023-10-07T07:57:50.733023+0000","last_became_active":"2023-10-07T07:57:50.726065+0000","last_became_peered":"2023-10-07T07:57:50.726065+0000","last_unstale":"2023-10-07T07:57:50.733023+0000","last_undegraded":"2023-10-07T07:57:50.733023+0000","last_fullsized":"2023-10-07T07:57:50.733023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:59:47.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735217+0000","last_change":"2023-10-07T07:57:50.734224+0000","last_active":"2023-10-07T07:57:50.735217+0000","last_peered":"2023-10-07T07:57:50.735217+0000","last_clean":"2023-10-07T07:57:50.735217+0000","last_became_active":"2023-10-07T07:57:50.729939+0000","last_became_peered":"2023-10-07T07:57:50.729939+0000","last_unstale":"2023-10-07T07:57:50.735217+0000","last_undegraded":"2023-10-07T07:57:50.735217+0000","last_fullsized":"2023-10-07T07:57:50.735217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:53:49.221151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735896+0000","last_change":"2023-10-07T07:57:50.730055+0000","last_active":"2023-10-07T07:57:50.735896+0000","last_peered":"2023-10-07T07:57:50.735896+0000","last_clean":"2023-10-07T07:57:50.735896+0000","last_became_active":"2023-10-07T07:57:50.729800+0000","last_became_peered":"2023-10-07T07:57:50.729800+0000","last_unstale":"2023-10-07T07:57:50.735896+0000","last_undegraded":"2023-10-07T07:57:50.735896+0000","last_fullsized":"2023-10-07T07:57:50.735896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:37:11.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725487+0000","last_change":"2023-10-07T07:57:50.718932+0000","last_active":"2023-10-07T07:57:50.725487+0000","last_peered":"2023-10-07T07:57:50.725487+0000","last_clean":"2023-10-07T07:57:50.725487+0000","last_became_active":"2023-10-07T07:57:50.718650+0000","last_became_peered":"2023-10-07T07:57:50.718650+0000","last_unstale":"2023-10-07T07:57:50.725487+0000","last_undegraded":"2023-10-07T07:57:50.725487+0000","last_fullsized":"2023-10-07T07:57:50.725487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:40.473036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.737108+0000","last_change":"2023-10-07T07:57:50.730279+0000","last_active":"2023-10-07T07:57:50.737108+0000","last_peered":"2023-10-07T07:57:50.737108+0000","last_clean":"2023-10-07T07:57:50.737108+0000","last_became_active":"2023-10-07T07:57:50.730004+0000","last_became_peered":"2023-10-07T07:57:50.730004+0000","last_unstale":"2023-10-07T07:57:50.737108+0000","last_undegraded":"2023-10-07T07:57:50.737108+0000","last_fullsized":"2023-10-07T07:57:50.737108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.728979+0000","last_change":"2023-10-07T07:57:50.720796+0000","last_active":"2023-10-07T07:57:50.728979+0000","last_peered":"2023-10-07T07:57:50.728979+0000","last_clean":"2023-10-07T07:57:50.728979+0000","last_became_active":"2023-10-07T07:57:50.720416+0000","last_became_peered":"2023-10-07T07:57:50.720416+0000","last_unstale":"2023-10-07T07:57:50.728979+0000","last_undegraded":"2023-10-07T07:57:50.728979+0000","last_fullsized":"2023-10-07T07:57:50.728979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:03:44.885267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743428+0000","last_change":"2023-10-07T07:57:50.741002+0000","last_active":"2023-10-07T07:57:50.743428+0000","last_peered":"2023-10-07T07:57:50.743428+0000","last_clean":"2023-10-07T07:57:50.743428+0000","last_became_active":"2023-10-07T07:57:50.740515+0000","last_became_peered":"2023-10-07T07:57:50.740515+0000","last_unstale":"2023-10-07T07:57:50.743428+0000","last_undegraded":"2023-10-07T07:57:50.743428+0000","last_fullsized":"2023-10-07T07:57:50.743428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:35:45.958299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.731000+0000","last_change":"2023-10-07T07:57:50.725365+0000","last_active":"2023-10-07T07:57:50.731000+0000","last_peered":"2023-10-07T07:57:50.731000+0000","last_clean":"2023-10-07T07:57:50.731000+0000","last_became_active":"2023-10-07T07:57:50.725076+0000","last_became_peered":"2023-10-07T07:57:50.725076+0000","last_unstale":"2023-10-07T07:57:50.731000+0000","last_undegraded":"2023-10-07T07:57:50.731000+0000","last_fullsized":"2023-10-07T07:57:50.731000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.736388+0000","last_change":"2023-10-07T07:57:50.722362+0000","last_active":"2023-10-07T07:57:50.736388+0000","last_peered":"2023-10-07T07:57:50.736388+0000","last_clean":"2023-10-07T07:57:50.736388+0000","last_became_active":"2023-10-07T07:57:50.722022+0000","last_became_peered":"2023-10-07T07:57:50.722022+0000","last_unstale":"2023-10-07T07:57:50.736388+0000","last_undegraded":"2023-10-07T07:57:50.736388+0000","last_fullsized":"2023-10-07T07:57:50.736388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:25:31.456924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742913+0000","last_change":"2023-10-07T07:57:49.726945+0000","last_active":"2023-10-07T07:57:50.742913+0000","last_peered":"2023-10-07T07:57:50.742913+0000","last_clean":"2023-10-07T07:57:50.742913+0000","last_became_active":"2023-10-07T07:57:49.726621+0000","last_became_peered":"2023-10-07T07:57:49.726621+0000","last_unstale":"2023-10-07T07:57:50.742913+0000","last_undegraded":"2023-10-07T07:57:50.742913+0000","last_fullsized":"2023-10-07T07:57:50.742913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741522+0000","last_change":"2023-10-07T07:57:50.740835+0000","last_active":"2023-10-07T07:57:50.741522+0000","last_peered":"2023-10-07T07:57:50.741522+0000","last_clean":"2023-10-07T07:57:50.741522+0000","last_became_active":"2023-10-07T07:57:50.740562+0000","last_became_peered":"2023-10-07T07:57:50.740562+0000","last_unstale":"2023-10-07T07:57:50.741522+0000","last_undegraded":"2023-10-07T07:57:50.741522+0000","last_fullsized":"2023-10-07T07:57:50.741522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:09:38.557090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738888+0000","last_change":"2023-10-07T07:57:49.725329+0000","last_active":"2023-10-07T07:57:50.738888+0000","last_peered":"2023-10-07T07:57:50.738888+0000","last_clean":"2023-10-07T07:57:50.738888+0000","last_became_active":"2023-10-07T07:57:49.724930+0000","last_became_peered":"2023-10-07T07:57:49.724930+0000","last_unstale":"2023-10-07T07:57:50.738888+0000","last_undegraded":"2023-10-07T07:57:50.738888+0000","last_fullsized":"2023-10-07T07:57:50.738888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725494+0000","last_change":"2023-10-07T07:57:50.724035+0000","last_active":"2023-10-07T07:57:50.725494+0000","last_peered":"2023-10-07T07:57:50.725494+0000","last_clean":"2023-10-07T07:57:50.725494+0000","last_became_active":"2023-10-07T07:57:50.723418+0000","last_became_peered":"2023-10-07T07:57:50.723418+0000","last_unstale":"2023-10-07T07:57:50.725494+0000","last_undegraded":"2023-10-07T07:57:50.725494+0000","last_fullsized":"2023-10-07T07:57:50.725494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741778+0000","last_change":"2023-10-07T07:57:49.727656+0000","last_active":"2023-10-07T07:57:50.741778+0000","last_peered":"2023-10-07T07:57:50.741778+0000","last_clean":"2023-10-07T07:57:50.741778+0000","last_became_active":"2023-10-07T07:57:49.727380+0000","last_became_peered":"2023-10-07T07:57:49.727380+0000","last_unstale":"2023-10-07T07:57:50.741778+0000","last_undegraded":"2023-10-07T07:57:50.741778+0000","last_fullsized":"2023-10-07T07:57:50.741778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:31.962724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732428+0000","last_change":"2023-10-07T07:57:50.727657+0000","last_active":"2023-10-07T07:57:50.732428+0000","last_peered":"2023-10-07T07:57:50.732428+0000","last_clean":"2023-10-07T07:57:50.732428+0000","last_became_active":"2023-10-07T07:57:50.727309+0000","last_became_peered":"2023-10-07T07:57:50.727309+0000","last_unstale":"2023-10-07T07:57:50.732428+0000","last_undegraded":"2023-10-07T07:57:50.732428+0000","last_fullsized":"2023-10-07T07:57:50.732428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743139+0000","last_change":"2023-10-07T07:57:49.729636+0000","last_active":"2023-10-07T07:57:50.743139+0000","last_peered":"2023-10-07T07:57:50.743139+0000","last_clean":"2023-10-07T07:57:50.743139+0000","last_became_active":"2023-10-07T07:57:49.725076+0000","last_became_peered":"2023-10-07T07:57:49.725076+0000","last_unstale":"2023-10-07T07:57:50.743139+0000","last_undegraded":"2023-10-07T07:57:50.743139+0000","last_fullsized":"2023-10-07T07:57:50.743139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:01.309236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743459+0000","last_change":"2023-10-07T07:57:50.742189+0000","last_active":"2023-10-07T07:57:50.743459+0000","last_peered":"2023-10-07T07:57:50.743459+0000","last_clean":"2023-10-07T07:57:50.743459+0000","last_became_active":"2023-10-07T07:57:50.741823+0000","last_became_peered":"2023-10-07T07:57:50.741823+0000","last_unstale":"2023-10-07T07:57:50.743459+0000","last_undegraded":"2023-10-07T07:57:50.743459+0000","last_fullsized":"2023-10-07T07:57:50.743459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:18:23.885325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738865+0000","last_change":"2023-10-07T07:57:49.724287+0000","last_active":"2023-10-07T07:57:50.738865+0000","last_peered":"2023-10-07T07:57:50.738865+0000","last_clean":"2023-10-07T07:57:50.738865+0000","last_became_active":"2023-10-07T07:57:49.723711+0000","last_became_peered":"2023-10-07T07:57:49.723711+0000","last_unstale":"2023-10-07T07:57:50.738865+0000","last_undegraded":"2023-10-07T07:57:50.738865+0000","last_fullsized":"2023-10-07T07:57:50.738865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726639+0000","last_change":"2023-10-07T07:57:50.723737+0000","last_active":"2023-10-07T07:57:50.726639+0000","last_peered":"2023-10-07T07:57:50.726639+0000","last_clean":"2023-10-07T07:57:50.726639+0000","last_became_active":"2023-10-07T07:57:50.723444+0000","last_became_peered":"2023-10-07T07:57:50.723444+0000","last_unstale":"2023-10-07T07:57:50.726639+0000","last_undegraded":"2023-10-07T07:57:50.726639+0000","last_fullsized":"2023-10-07T07:57:50.726639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.744159+0000","last_change":"2023-10-07T07:57:49.737749+0000","last_active":"2023-10-07T07:57:50.744159+0000","last_peered":"2023-10-07T07:57:50.744159+0000","last_clean":"2023-10-07T07:57:50.744159+0000","last_became_active":"2023-10-07T07:57:49.737477+0000","last_became_peered":"2023-10-07T07:57:49.737477+0000","last_unstale":"2023-10-07T07:57:50.744159+0000","last_undegraded":"2023-10-07T07:57:50.744159+0000","last_fullsized":"2023-10-07T07:57:50.744159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:10.280748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734802+0000","last_change":"2023-10-07T07:57:50.730239+0000","last_active":"2023-10-07T07:57:50.734802+0000","last_peered":"2023-10-07T07:57:50.734802+0000","last_clean":"2023-10-07T07:57:50.734802+0000","last_became_active":"2023-10-07T07:57:50.729972+0000","last_became_peered":"2023-10-07T07:57:50.729972+0000","last_unstale":"2023-10-07T07:57:50.734802+0000","last_undegraded":"2023-10-07T07:57:50.734802+0000","last_fullsized":"2023-10-07T07:57:50.734802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:03:06.838727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711739+0000","last_change":"2023-10-07T07:57:49.728260+0000","last_active":"2023-10-07T07:57:50.711739+0000","last_peered":"2023-10-07T07:57:50.711739+0000","last_clean":"2023-10-07T07:57:50.711739+0000","last_became_active":"2023-10-07T07:57:49.728019+0000","last_became_peered":"2023-10-07T07:57:49.728019+0000","last_unstale":"2023-10-07T07:57:50.711739+0000","last_undegraded":"2023-10-07T07:57:50.711739+0000","last_fullsized":"2023-10-07T07:57:50.711739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726987+0000","last_change":"2023-10-07T07:57:50.723700+0000","last_active":"2023-10-07T07:57:50.726987+0000","last_peered":"2023-10-07T07:57:50.726987+0000","last_clean":"2023-10-07T07:57:50.726987+0000","last_became_active":"2023-10-07T07:57:50.723420+0000","last_became_peered":"2023-10-07T07:57:50.723420+0000","last_unstale":"2023-10-07T07:57:50.726987+0000","last_undegraded":"2023-10-07T07:57:50.726987+0000","last_fullsized":"2023-10-07T07:57:50.726987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:26:00.336772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712589+0000","last_change":"2023-10-07T07:57:49.732565+0000","last_active":"2023-10-07T07:57:50.712589+0000","last_peered":"2023-10-07T07:57:50.712589+0000","last_clean":"2023-10-07T07:57:50.712589+0000","last_became_active":"2023-10-07T07:57:49.732250+0000","last_became_peered":"2023-10-07T07:57:49.732250+0000","last_unstale":"2023-10-07T07:57:50.712589+0000","last_undegraded":"2023-10-07T07:57:50.712589+0000","last_fullsized":"2023-10-07T07:57:50.712589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727015+0000","last_change":"2023-10-07T07:57:50.722227+0000","last_active":"2023-10-07T07:57:50.727015+0000","last_peered":"2023-10-07T07:57:50.727015+0000","last_clean":"2023-10-07T07:57:50.727015+0000","last_became_active":"2023-10-07T07:57:50.721953+0000","last_became_peered":"2023-10-07T07:57:50.721953+0000","last_unstale":"2023-10-07T07:57:50.727015+0000","last_undegraded":"2023-10-07T07:57:50.727015+0000","last_fullsized":"2023-10-07T07:57:50.727015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:50:56.892918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.709932+0000","last_change":"2023-10-07T07:57:49.720571+0000","last_active":"2023-10-07T07:57:50.709932+0000","last_peered":"2023-10-07T07:57:50.709932+0000","last_clean":"2023-10-07T07:57:50.709932+0000","last_became_active":"2023-10-07T07:57:49.720226+0000","last_became_peered":"2023-10-07T07:57:49.720226+0000","last_unstale":"2023-10-07T07:57:50.709932+0000","last_undegraded":"2023-10-07T07:57:50.709932+0000","last_fullsized":"2023-10-07T07:57:50.709932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740842+0000","last_change":"2023-10-07T07:57:50.724121+0000","last_active":"2023-10-07T07:57:50.740842+0000","last_peered":"2023-10-07T07:57:50.740842+0000","last_clean":"2023-10-07T07:57:50.740842+0000","last_became_active":"2023-10-07T07:57:50.723789+0000","last_became_peered":"2023-10-07T07:57:50.723789+0000","last_unstale":"2023-10-07T07:57:50.740842+0000","last_undegraded":"2023-10-07T07:57:50.740842+0000","last_fullsized":"2023-10-07T07:57:50.740842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:32.108899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.720435+0000","last_change":"2023-10-07T07:57:49.733414+0000","last_active":"2023-10-07T07:57:50.720435+0000","last_peered":"2023-10-07T07:57:50.720435+0000","last_clean":"2023-10-07T07:57:50.720435+0000","last_became_active":"2023-10-07T07:57:49.732736+0000","last_became_peered":"2023-10-07T07:57:49.732736+0000","last_unstale":"2023-10-07T07:57:50.720435+0000","last_undegraded":"2023-10-07T07:57:50.720435+0000","last_fullsized":"2023-10-07T07:57:50.720435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:49.692513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727348+0000","last_change":"2023-10-07T07:57:50.724743+0000","last_active":"2023-10-07T07:57:50.727348+0000","last_peered":"2023-10-07T07:57:50.727348+0000","last_clean":"2023-10-07T07:57:50.727348+0000","last_became_active":"2023-10-07T07:57:50.724501+0000","last_became_peered":"2023-10-07T07:57:50.724501+0000","last_unstale":"2023-10-07T07:57:50.727348+0000","last_undegraded":"2023-10-07T07:57:50.727348+0000","last_fullsized":"2023-10-07T07:57:50.727348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:03:23.294266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738823+0000","last_change":"2023-10-07T07:57:49.721843+0000","last_active":"2023-10-07T07:57:50.738823+0000","last_peered":"2023-10-07T07:57:50.738823+0000","last_clean":"2023-10-07T07:57:50.738823+0000","last_became_active":"2023-10-07T07:57:49.721533+0000","last_became_peered":"2023-10-07T07:57:49.721533+0000","last_unstale":"2023-10-07T07:57:50.738823+0000","last_undegraded":"2023-10-07T07:57:50.738823+0000","last_fullsized":"2023-10-07T07:57:50.738823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729050+0000","last_change":"2023-10-07T07:57:50.724096+0000","last_active":"2023-10-07T07:57:50.729050+0000","last_peered":"2023-10-07T07:57:50.729050+0000","last_clean":"2023-10-07T07:57:50.729050+0000","last_became_active":"2023-10-07T07:57:50.723782+0000","last_became_peered":"2023-10-07T07:57:50.723782+0000","last_unstale":"2023-10-07T07:57:50.729050+0000","last_undegraded":"2023-10-07T07:57:50.729050+0000","last_fullsized":"2023-10-07T07:57:50.729050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:59:47.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.746945+0000","last_change":"2023-10-07T07:57:49.720185+0000","last_active":"2023-10-07T07:57:50.746945+0000","last_peered":"2023-10-07T07:57:50.746945+0000","last_clean":"2023-10-07T07:57:50.746945+0000","last_became_active":"2023-10-07T07:57:49.719880+0000","last_became_peered":"2023-10-07T07:57:49.719880+0000","last_unstale":"2023-10-07T07:57:50.746945+0000","last_undegraded":"2023-10-07T07:57:50.746945+0000","last_fullsized":"2023-10-07T07:57:50.746945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:53:25.421958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732557+0000","last_change":"2023-10-07T07:57:50.726280+0000","last_active":"2023-10-07T07:57:50.732557+0000","last_peered":"2023-10-07T07:57:50.732557+0000","last_clean":"2023-10-07T07:57:50.732557+0000","last_became_active":"2023-10-07T07:57:50.725649+0000","last_became_peered":"2023-10-07T07:57:50.725649+0000","last_unstale":"2023-10-07T07:57:50.732557+0000","last_undegraded":"2023-10-07T07:57:50.732557+0000","last_fullsized":"2023-10-07T07:57:50.732557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:22:50.873696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739735+0000","last_change":"2023-10-07T07:57:49.732331+0000","last_active":"2023-10-07T07:57:50.739735+0000","last_peered":"2023-10-07T07:57:50.739735+0000","last_clean":"2023-10-07T07:57:50.739735+0000","last_became_active":"2023-10-07T07:57:49.732085+0000","last_became_peered":"2023-10-07T07:57:49.732085+0000","last_unstale":"2023-10-07T07:57:50.739735+0000","last_undegraded":"2023-10-07T07:57:50.739735+0000","last_fullsized":"2023-10-07T07:57:50.739735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727938+0000","last_change":"2023-10-07T07:57:50.724196+0000","last_active":"2023-10-07T07:57:50.727938+0000","last_peered":"2023-10-07T07:57:50.727938+0000","last_clean":"2023-10-07T07:57:50.727938+0000","last_became_active":"2023-10-07T07:57:50.723911+0000","last_became_peered":"2023-10-07T07:57:50.723911+0000","last_unstale":"2023-10-07T07:57:50.727938+0000","last_undegraded":"2023-10-07T07:57:50.727938+0000","last_fullsized":"2023-10-07T07:57:50.727938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:42.757604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712455+0000","last_change":"2023-10-07T07:57:49.734820+0000","last_active":"2023-10-07T07:57:50.712455+0000","last_peered":"2023-10-07T07:57:50.712455+0000","last_clean":"2023-10-07T07:57:50.712455+0000","last_became_active":"2023-10-07T07:57:49.732635+0000","last_became_peered":"2023-10-07T07:57:49.732635+0000","last_unstale":"2023-10-07T07:57:50.712455+0000","last_undegraded":"2023-10-07T07:57:50.712455+0000","last_fullsized":"2023-10-07T07:57:50.712455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:36:22.098994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.743306+0000","last_change":"2023-10-07T07:57:50.740602+0000","last_active":"2023-10-07T07:57:50.743306+0000","last_peered":"2023-10-07T07:57:50.743306+0000","last_clean":"2023-10-07T07:57:50.743306+0000","last_became_active":"2023-10-07T07:57:50.740273+0000","last_became_peered":"2023-10-07T07:57:50.740273+0000","last_unstale":"2023-10-07T07:57:50.743306+0000","last_undegraded":"2023-10-07T07:57:50.743306+0000","last_fullsized":"2023-10-07T07:57:50.743306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:33:31.464165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.716494+0000","last_change":"2023-10-07T07:57:49.738688+0000","last_active":"2023-10-07T07:57:50.716494+0000","last_peered":"2023-10-07T07:57:50.716494+0000","last_clean":"2023-10-07T07:57:50.716494+0000","last_became_active":"2023-10-07T07:57:49.738387+0000","last_became_peered":"2023-10-07T07:57:49.738387+0000","last_unstale":"2023-10-07T07:57:50.716494+0000","last_undegraded":"2023-10-07T07:57:50.716494+0000","last_fullsized":"2023-10-07T07:57:50.716494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726565+0000","last_change":"2023-10-07T07:57:50.723301+0000","last_active":"2023-10-07T07:57:50.726565+0000","last_peered":"2023-10-07T07:57:50.726565+0000","last_clean":"2023-10-07T07:57:50.726565+0000","last_became_active":"2023-10-07T07:57:50.722920+0000","last_became_peered":"2023-10-07T07:57:50.722920+0000","last_unstale":"2023-10-07T07:57:50.726565+0000","last_undegraded":"2023-10-07T07:57:50.726565+0000","last_fullsized":"2023-10-07T07:57:50.726565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:32.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739604+0000","last_change":"2023-10-07T07:57:49.719344+0000","last_active":"2023-10-07T07:57:50.739604+0000","last_peered":"2023-10-07T07:57:50.739604+0000","last_clean":"2023-10-07T07:57:50.739604+0000","last_became_active":"2023-10-07T07:57:49.719053+0000","last_became_peered":"2023-10-07T07:57:49.719053+0000","last_unstale":"2023-10-07T07:57:50.739604+0000","last_undegraded":"2023-10-07T07:57:50.739604+0000","last_fullsized":"2023-10-07T07:57:50.739604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:32:15.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729136+0000","last_change":"2023-10-07T07:57:50.724500+0000","last_active":"2023-10-07T07:57:50.729136+0000","last_peered":"2023-10-07T07:57:50.729136+0000","last_clean":"2023-10-07T07:57:50.729136+0000","last_became_active":"2023-10-07T07:57:50.724175+0000","last_became_peered":"2023-10-07T07:57:50.724175+0000","last_unstale":"2023-10-07T07:57:50.729136+0000","last_undegraded":"2023-10-07T07:57:50.729136+0000","last_fullsized":"2023-10-07T07:57:50.729136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:47:49.499637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739659+0000","last_change":"2023-10-07T07:57:49.733130+0000","last_active":"2023-10-07T07:57:50.739659+0000","last_peered":"2023-10-07T07:57:50.739659+0000","last_clean":"2023-10-07T07:57:50.739659+0000","last_became_active":"2023-10-07T07:57:49.732810+0000","last_became_peered":"2023-10-07T07:57:49.732810+0000","last_unstale":"2023-10-07T07:57:50.739659+0000","last_undegraded":"2023-10-07T07:57:50.739659+0000","last_fullsized":"2023-10-07T07:57:50.739659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:32:15.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741983+0000","last_change":"2023-10-07T07:57:50.731887+0000","last_active":"2023-10-07T07:57:50.741983+0000","last_peered":"2023-10-07T07:57:50.741983+0000","last_clean":"2023-10-07T07:57:50.741983+0000","last_became_active":"2023-10-07T07:57:50.731535+0000","last_became_peered":"2023-10-07T07:57:50.731535+0000","last_unstale":"2023-10-07T07:57:50.741983+0000","last_undegraded":"2023-10-07T07:57:50.741983+0000","last_fullsized":"2023-10-07T07:57:50.741983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:40.473036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711292+0000","last_change":"2023-10-07T07:57:49.722522+0000","last_active":"2023-10-07T07:57:50.711292+0000","last_peered":"2023-10-07T07:57:50.711292+0000","last_clean":"2023-10-07T07:57:50.711292+0000","last_became_active":"2023-10-07T07:57:49.722225+0000","last_became_peered":"2023-10-07T07:57:49.722225+0000","last_unstale":"2023-10-07T07:57:50.711292+0000","last_undegraded":"2023-10-07T07:57:50.711292+0000","last_fullsized":"2023-10-07T07:57:50.711292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.719014+0000","last_change":"2023-10-07T07:57:50.717165+0000","last_active":"2023-10-07T07:57:50.719014+0000","last_peered":"2023-10-07T07:57:50.719014+0000","last_clean":"2023-10-07T07:57:50.719014+0000","last_became_active":"2023-10-07T07:57:50.716840+0000","last_became_peered":"2023-10-07T07:57:50.716840+0000","last_unstale":"2023-10-07T07:57:50.719014+0000","last_undegraded":"2023-10-07T07:57:50.719014+0000","last_fullsized":"2023-10-07T07:57:50.719014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:50.698077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712289+0000","last_change":"2023-10-07T07:57:49.735497+0000","last_active":"2023-10-07T07:57:50.712289+0000","last_peered":"2023-10-07T07:57:50.712289+0000","last_clean":"2023-10-07T07:57:50.712289+0000","last_became_active":"2023-10-07T07:57:49.735195+0000","last_became_peered":"2023-10-07T07:57:49.735195+0000","last_unstale":"2023-10-07T07:57:50.712289+0000","last_undegraded":"2023-10-07T07:57:50.712289+0000","last_fullsized":"2023-10-07T07:57:50.712289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:06:39.467472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742402+0000","last_change":"2023-10-07T07:57:50.741594+0000","last_active":"2023-10-07T07:57:50.742402+0000","last_peered":"2023-10-07T07:57:50.742402+0000","last_clean":"2023-10-07T07:57:50.742402+0000","last_became_active":"2023-10-07T07:57:50.741331+0000","last_became_peered":"2023-10-07T07:57:50.741331+0000","last_unstale":"2023-10-07T07:57:50.742402+0000","last_undegraded":"2023-10-07T07:57:50.742402+0000","last_fullsized":"2023-10-07T07:57:50.742402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:26:00.336772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733078+0000","last_change":"2023-10-07T07:57:50.726796+0000","last_active":"2023-10-07T07:57:50.733078+0000","last_peered":"2023-10-07T07:57:50.733078+0000","last_clean":"2023-10-07T07:57:50.733078+0000","last_became_active":"2023-10-07T07:57:50.725206+0000","last_became_peered":"2023-10-07T07:57:50.725206+0000","last_unstale":"2023-10-07T07:57:50.733078+0000","last_undegraded":"2023-10-07T07:57:50.733078+0000","last_fullsized":"2023-10-07T07:57:50.733078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:46:16.424520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.714634+0000","last_change":"2023-10-07T07:57:49.722494+0000","last_active":"2023-10-07T07:57:50.714634+0000","last_peered":"2023-10-07T07:57:50.714634+0000","last_clean":"2023-10-07T07:57:50.714634+0000","last_became_active":"2023-10-07T07:57:49.722237+0000","last_became_peered":"2023-10-07T07:57:49.722237+0000","last_unstale":"2023-10-07T07:57:50.714634+0000","last_undegraded":"2023-10-07T07:57:50.714634+0000","last_fullsized":"2023-10-07T07:57:50.714634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722017+0000","last_change":"2023-10-07T07:57:50.718217+0000","last_active":"2023-10-07T07:57:50.722017+0000","last_peered":"2023-10-07T07:57:50.722017+0000","last_clean":"2023-10-07T07:57:50.722017+0000","last_became_active":"2023-10-07T07:57:50.717791+0000","last_became_peered":"2023-10-07T07:57:50.717791+0000","last_unstale":"2023-10-07T07:57:50.722017+0000","last_undegraded":"2023-10-07T07:57:50.722017+0000","last_fullsized":"2023-10-07T07:57:50.722017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:59:42.757604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711986+0000","last_change":"2023-10-07T07:57:49.733338+0000","last_active":"2023-10-07T07:57:50.711986+0000","last_peered":"2023-10-07T07:57:50.711986+0000","last_clean":"2023-10-07T07:57:50.711986+0000","last_became_active":"2023-10-07T07:57:49.732478+0000","last_became_peered":"2023-10-07T07:57:49.732478+0000","last_unstale":"2023-10-07T07:57:50.711986+0000","last_undegraded":"2023-10-07T07:57:50.711986+0000","last_fullsized":"2023-10-07T07:57:50.711986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:01.309236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.709777+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T07:57:50.709777+0000","last_peered":"2023-10-07T07:57:50.709777+0000","last_clean":"2023-10-07T07:57:50.709777+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T07:57:50.709777+0000","last_undegraded":"2023-10-07T07:57:50.709777+0000","last_fullsized":"2023-10-07T07:57:50.709777+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727905+0000","last_change":"2023-10-07T07:57:50.723683+0000","last_active":"2023-10-07T07:57:50.727905+0000","last_peered":"2023-10-07T07:57:50.727905+0000","last_clean":"2023-10-07T07:57:50.727905+0000","last_became_active":"2023-10-07T07:57:50.723408+0000","last_became_peered":"2023-10-07T07:57:50.723408+0000","last_unstale":"2023-10-07T07:57:50.727905+0000","last_undegraded":"2023-10-07T07:57:50.727905+0000","last_fullsized":"2023-10-07T07:57:50.727905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:11.286312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712644+0000","last_change":"2023-10-07T07:57:49.733672+0000","last_active":"2023-10-07T07:57:50.712644+0000","last_peered":"2023-10-07T07:57:50.712644+0000","last_clean":"2023-10-07T07:57:50.712644+0000","last_became_active":"2023-10-07T07:57:49.733375+0000","last_became_peered":"2023-10-07T07:57:49.733375+0000","last_unstale":"2023-10-07T07:57:50.712644+0000","last_undegraded":"2023-10-07T07:57:50.712644+0000","last_fullsized":"2023-10-07T07:57:50.712644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:17:36.924302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.713200+0000","last_change":"2023-10-07T07:57:49.730927+0000","last_active":"2023-10-07T07:57:50.713200+0000","last_peered":"2023-10-07T07:57:50.713200+0000","last_clean":"2023-10-07T07:57:50.713200+0000","last_became_active":"2023-10-07T07:57:49.727008+0000","last_became_peered":"2023-10-07T07:57:49.727008+0000","last_unstale":"2023-10-07T07:57:50.713200+0000","last_undegraded":"2023-10-07T07:57:50.713200+0000","last_fullsized":"2023-10-07T07:57:50.713200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727827+0000","last_change":"2023-10-07T07:57:50.723292+0000","last_active":"2023-10-07T07:57:50.727827+0000","last_peered":"2023-10-07T07:57:50.727827+0000","last_clean":"2023-10-07T07:57:50.727827+0000","last_became_active":"2023-10-07T07:57:50.722388+0000","last_became_peered":"2023-10-07T07:57:50.722388+0000","last_unstale":"2023-10-07T07:57:50.727827+0000","last_undegraded":"2023-10-07T07:57:50.727827+0000","last_fullsized":"2023-10-07T07:57:50.727827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:53:57.122070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T07:57:53.552332+0000","last_change":"2023-10-07T07:57:49.736910+0000","last_active":"2023-10-07T07:57:53.552332+0000","last_peered":"2023-10-07T07:57:53.552332+0000","last_clean":"2023-10-07T07:57:53.552332+0000","last_became_active":"2023-10-07T07:57:49.736645+0000","last_became_peered":"2023-10-07T07:57:49.736645+0000","last_unstale":"2023-10-07T07:57:53.552332+0000","last_undegraded":"2023-10-07T07:57:53.552332+0000","last_fullsized":"2023-10-07T07:57:53.552332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:31:32.437158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.726999+0000","last_change":"2023-10-07T07:57:50.721660+0000","last_active":"2023-10-07T07:57:50.726999+0000","last_peered":"2023-10-07T07:57:50.726999+0000","last_clean":"2023-10-07T07:57:50.726999+0000","last_became_active":"2023-10-07T07:57:50.721390+0000","last_became_peered":"2023-10-07T07:57:50.721390+0000","last_unstale":"2023-10-07T07:57:50.726999+0000","last_undegraded":"2023-10-07T07:57:50.726999+0000","last_fullsized":"2023-10-07T07:57:50.726999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:11.286312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.745865+0000","last_change":"2023-10-07T07:57:49.737869+0000","last_active":"2023-10-07T07:57:50.745865+0000","last_peered":"2023-10-07T07:57:50.745865+0000","last_clean":"2023-10-07T07:57:50.745865+0000","last_became_active":"2023-10-07T07:57:49.737571+0000","last_became_peered":"2023-10-07T07:57:49.737571+0000","last_unstale":"2023-10-07T07:57:50.745865+0000","last_undegraded":"2023-10-07T07:57:50.745865+0000","last_fullsized":"2023-10-07T07:57:50.745865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:49.692513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722039+0000","last_change":"2023-10-07T07:57:50.718352+0000","last_active":"2023-10-07T07:57:50.722039+0000","last_peered":"2023-10-07T07:57:50.722039+0000","last_clean":"2023-10-07T07:57:50.722039+0000","last_became_active":"2023-10-07T07:57:50.718091+0000","last_became_peered":"2023-10-07T07:57:50.718091+0000","last_unstale":"2023-10-07T07:57:50.722039+0000","last_undegraded":"2023-10-07T07:57:50.722039+0000","last_fullsized":"2023-10-07T07:57:50.722039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:37:11.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.714964+0000","last_change":"2023-10-07T07:57:49.730601+0000","last_active":"2023-10-07T07:57:50.714964+0000","last_peered":"2023-10-07T07:57:50.714964+0000","last_clean":"2023-10-07T07:57:50.714964+0000","last_became_active":"2023-10-07T07:57:49.730370+0000","last_became_peered":"2023-10-07T07:57:49.730370+0000","last_unstale":"2023-10-07T07:57:50.714964+0000","last_undegraded":"2023-10-07T07:57:50.714964+0000","last_fullsized":"2023-10-07T07:57:50.714964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729113+0000","last_change":"2023-10-07T07:57:50.723162+0000","last_active":"2023-10-07T07:57:50.729113+0000","last_peered":"2023-10-07T07:57:50.729113+0000","last_clean":"2023-10-07T07:57:50.729113+0000","last_became_active":"2023-10-07T07:57:50.722848+0000","last_became_peered":"2023-10-07T07:57:50.722848+0000","last_unstale":"2023-10-07T07:57:50.729113+0000","last_undegraded":"2023-10-07T07:57:50.729113+0000","last_fullsized":"2023-10-07T07:57:50.729113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712342+0000","last_change":"2023-10-07T07:57:49.731859+0000","last_active":"2023-10-07T07:57:50.712342+0000","last_peered":"2023-10-07T07:57:50.712342+0000","last_clean":"2023-10-07T07:57:50.712342+0000","last_became_active":"2023-10-07T07:57:49.731613+0000","last_became_peered":"2023-10-07T07:57:49.731613+0000","last_unstale":"2023-10-07T07:57:50.712342+0000","last_undegraded":"2023-10-07T07:57:50.712342+0000","last_fullsized":"2023-10-07T07:57:50.712342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:09:46.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.727214+0000","last_change":"2023-10-07T07:57:50.723167+0000","last_active":"2023-10-07T07:57:50.727214+0000","last_peered":"2023-10-07T07:57:50.727214+0000","last_clean":"2023-10-07T07:57:50.727214+0000","last_became_active":"2023-10-07T07:57:50.722863+0000","last_became_peered":"2023-10-07T07:57:50.722863+0000","last_unstale":"2023-10-07T07:57:50.727214+0000","last_undegraded":"2023-10-07T07:57:50.727214+0000","last_fullsized":"2023-10-07T07:57:50.727214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.711876+0000","last_change":"2023-10-07T07:57:49.725087+0000","last_active":"2023-10-07T07:57:50.711876+0000","last_peered":"2023-10-07T07:57:50.711876+0000","last_clean":"2023-10-07T07:57:50.711876+0000","last_became_active":"2023-10-07T07:57:49.724027+0000","last_became_peered":"2023-10-07T07:57:49.724027+0000","last_unstale":"2023-10-07T07:57:50.711876+0000","last_undegraded":"2023-10-07T07:57:50.711876+0000","last_fullsized":"2023-10-07T07:57:50.711876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.722299+0000","last_change":"2023-10-07T07:57:50.717348+0000","last_active":"2023-10-07T07:57:50.722299+0000","last_peered":"2023-10-07T07:57:50.722299+0000","last_clean":"2023-10-07T07:57:50.722299+0000","last_became_active":"2023-10-07T07:57:50.716959+0000","last_became_peered":"2023-10-07T07:57:50.716959+0000","last_unstale":"2023-10-07T07:57:50.722299+0000","last_undegraded":"2023-10-07T07:57:50.722299+0000","last_fullsized":"2023-10-07T07:57:50.722299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.742144+0000","last_change":"2023-10-07T07:57:49.732599+0000","last_active":"2023-10-07T07:57:50.742144+0000","last_peered":"2023-10-07T07:57:50.742144+0000","last_clean":"2023-10-07T07:57:50.742144+0000","last_became_active":"2023-10-07T07:57:49.732088+0000","last_became_peered":"2023-10-07T07:57:49.732088+0000","last_unstale":"2023-10-07T07:57:50.742144+0000","last_undegraded":"2023-10-07T07:57:50.742144+0000","last_fullsized":"2023-10-07T07:57:50.742144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:17:31.103335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729208+0000","last_change":"2023-10-07T07:57:50.725007+0000","last_active":"2023-10-07T07:57:50.729208+0000","last_peered":"2023-10-07T07:57:50.729208+0000","last_clean":"2023-10-07T07:57:50.729208+0000","last_became_active":"2023-10-07T07:57:50.723895+0000","last_became_peered":"2023-10-07T07:57:50.723895+0000","last_unstale":"2023-10-07T07:57:50.729208+0000","last_undegraded":"2023-10-07T07:57:50.729208+0000","last_fullsized":"2023-10-07T07:57:50.729208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:13:37.457724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.712397+0000","last_change":"2023-10-07T07:57:49.725916+0000","last_active":"2023-10-07T07:57:50.712397+0000","last_peered":"2023-10-07T07:57:50.712397+0000","last_clean":"2023-10-07T07:57:50.712397+0000","last_became_active":"2023-10-07T07:57:49.725648+0000","last_became_peered":"2023-10-07T07:57:49.725648+0000","last_unstale":"2023-10-07T07:57:50.712397+0000","last_undegraded":"2023-10-07T07:57:50.712397+0000","last_fullsized":"2023-10-07T07:57:50.712397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.736361+0000","last_change":"2023-10-07T07:57:50.720453+0000","last_active":"2023-10-07T07:57:50.736361+0000","last_peered":"2023-10-07T07:57:50.736361+0000","last_clean":"2023-10-07T07:57:50.736361+0000","last_became_active":"2023-10-07T07:57:50.719863+0000","last_became_peered":"2023-10-07T07:57:50.719863+0000","last_unstale":"2023-10-07T07:57:50.736361+0000","last_undegraded":"2023-10-07T07:57:50.736361+0000","last_fullsized":"2023-10-07T07:57:50.736361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:21:30.452139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739318+0000","last_change":"2023-10-07T07:57:49.729606+0000","last_active":"2023-10-07T07:57:50.739318+0000","last_peered":"2023-10-07T07:57:50.739318+0000","last_clean":"2023-10-07T07:57:50.739318+0000","last_became_active":"2023-10-07T07:57:49.729372+0000","last_became_peered":"2023-10-07T07:57:49.729372+0000","last_unstale":"2023-10-07T07:57:50.739318+0000","last_undegraded":"2023-10-07T07:57:50.739318+0000","last_fullsized":"2023-10-07T07:57:50.739318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:05:41.682144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735545+0000","last_change":"2023-10-07T07:57:50.734705+0000","last_active":"2023-10-07T07:57:50.735545+0000","last_peered":"2023-10-07T07:57:50.735545+0000","last_clean":"2023-10-07T07:57:50.735545+0000","last_became_active":"2023-10-07T07:57:50.733706+0000","last_became_peered":"2023-10-07T07:57:50.733706+0000","last_unstale":"2023-10-07T07:57:50.735545+0000","last_undegraded":"2023-10-07T07:57:50.735545+0000","last_fullsized":"2023-10-07T07:57:50.735545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:56:34.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.739193+0000","last_change":"2023-10-07T07:57:49.724649+0000","last_active":"2023-10-07T07:57:50.739193+0000","last_peered":"2023-10-07T07:57:50.739193+0000","last_clean":"2023-10-07T07:57:50.739193+0000","last_became_active":"2023-10-07T07:57:49.724368+0000","last_became_peered":"2023-10-07T07:57:49.724368+0000","last_unstale":"2023-10-07T07:57:50.739193+0000","last_undegraded":"2023-10-07T07:57:50.739193+0000","last_fullsized":"2023-10-07T07:57:50.739193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:48.687728+0000","last_clean_scrub_stamp":"2023-10-07T07:57:48.687728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:58:10.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733770+0000","last_change":"2023-10-07T07:57:50.730366+0000","last_active":"2023-10-07T07:57:50.733770+0000","last_peered":"2023-10-07T07:57:50.733770+0000","last_clean":"2023-10-07T07:57:50.733770+0000","last_became_active":"2023-10-07T07:57:50.730012+0000","last_became_peered":"2023-10-07T07:57:50.730012+0000","last_unstale":"2023-10-07T07:57:50.733770+0000","last_undegraded":"2023-10-07T07:57:50.733770+0000","last_fullsized":"2023-10-07T07:57:50.733770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:15:16.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740610+0000","last_change":"2023-10-07T07:57:50.721978+0000","last_active":"2023-10-07T07:57:50.740610+0000","last_peered":"2023-10-07T07:57:50.740610+0000","last_clean":"2023-10-07T07:57:50.740610+0000","last_became_active":"2023-10-07T07:57:50.719507+0000","last_became_peered":"2023-10-07T07:57:50.719507+0000","last_unstale":"2023-10-07T07:57:50.740610+0000","last_undegraded":"2023-10-07T07:57:50.740610+0000","last_fullsized":"2023-10-07T07:57:50.740610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:32.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.730933+0000","last_change":"2023-10-07T07:57:50.724692+0000","last_active":"2023-10-07T07:57:50.730933+0000","last_peered":"2023-10-07T07:57:50.730933+0000","last_clean":"2023-10-07T07:57:50.730933+0000","last_became_active":"2023-10-07T07:57:50.724411+0000","last_became_peered":"2023-10-07T07:57:50.724411+0000","last_unstale":"2023-10-07T07:57:50.730933+0000","last_undegraded":"2023-10-07T07:57:50.730933+0000","last_fullsized":"2023-10-07T07:57:50.730933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:38:02.314800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.732488+0000","last_change":"2023-10-07T07:57:50.727607+0000","last_active":"2023-10-07T07:57:50.732488+0000","last_peered":"2023-10-07T07:57:50.732488+0000","last_clean":"2023-10-07T07:57:50.732488+0000","last_became_active":"2023-10-07T07:57:50.724048+0000","last_became_peered":"2023-10-07T07:57:50.724048+0000","last_unstale":"2023-10-07T07:57:50.732488+0000","last_undegraded":"2023-10-07T07:57:50.732488+0000","last_fullsized":"2023-10-07T07:57:50.732488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:38:14.739161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.734147+0000","last_change":"2023-10-07T07:57:50.726261+0000","last_active":"2023-10-07T07:57:50.734147+0000","last_peered":"2023-10-07T07:57:50.734147+0000","last_clean":"2023-10-07T07:57:50.734147+0000","last_became_active":"2023-10-07T07:57:50.725636+0000","last_became_peered":"2023-10-07T07:57:50.725636+0000","last_unstale":"2023-10-07T07:57:50.734147+0000","last_undegraded":"2023-10-07T07:57:50.734147+0000","last_fullsized":"2023-10-07T07:57:50.734147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:56:34.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.738656+0000","last_change":"2023-10-07T07:57:50.735497+0000","last_active":"2023-10-07T07:57:50.738656+0000","last_peered":"2023-10-07T07:57:50.738656+0000","last_clean":"2023-10-07T07:57:50.738656+0000","last_became_active":"2023-10-07T07:57:50.735172+0000","last_became_peered":"2023-10-07T07:57:50.735172+0000","last_unstale":"2023-10-07T07:57:50.738656+0000","last_undegraded":"2023-10-07T07:57:50.738656+0000","last_fullsized":"2023-10-07T07:57:50.738656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725429+0000","last_change":"2023-10-07T07:57:50.718981+0000","last_active":"2023-10-07T07:57:50.725429+0000","last_peered":"2023-10-07T07:57:50.725429+0000","last_clean":"2023-10-07T07:57:50.725429+0000","last_became_active":"2023-10-07T07:57:50.718699+0000","last_became_peered":"2023-10-07T07:57:50.718699+0000","last_unstale":"2023-10-07T07:57:50.725429+0000","last_undegraded":"2023-10-07T07:57:50.725429+0000","last_fullsized":"2023-10-07T07:57:50.725429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:36:23.104558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.740872+0000","last_change":"2023-10-07T07:57:50.724708+0000","last_active":"2023-10-07T07:57:50.740872+0000","last_peered":"2023-10-07T07:57:50.740872+0000","last_clean":"2023-10-07T07:57:50.740872+0000","last_became_active":"2023-10-07T07:57:50.724343+0000","last_became_peered":"2023-10-07T07:57:50.724343+0000","last_unstale":"2023-10-07T07:57:50.740872+0000","last_undegraded":"2023-10-07T07:57:50.740872+0000","last_fullsized":"2023-10-07T07:57:50.740872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:38:53.980034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.725505+0000","last_change":"2023-10-07T07:57:50.724233+0000","last_active":"2023-10-07T07:57:50.725505+0000","last_peered":"2023-10-07T07:57:50.725505+0000","last_clean":"2023-10-07T07:57:50.725505+0000","last_became_active":"2023-10-07T07:57:50.723770+0000","last_became_peered":"2023-10-07T07:57:50.723770+0000","last_unstale":"2023-10-07T07:57:50.725505+0000","last_undegraded":"2023-10-07T07:57:50.725505+0000","last_fullsized":"2023-10-07T07:57:50.725505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:22:50.873696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.729190+0000","last_change":"2023-10-07T07:57:50.724839+0000","last_active":"2023-10-07T07:57:50.729190+0000","last_peered":"2023-10-07T07:57:50.729190+0000","last_clean":"2023-10-07T07:57:50.729190+0000","last_became_active":"2023-10-07T07:57:50.724489+0000","last_became_peered":"2023-10-07T07:57:50.724489+0000","last_unstale":"2023-10-07T07:57:50.729190+0000","last_undegraded":"2023-10-07T07:57:50.729190+0000","last_fullsized":"2023-10-07T07:57:50.729190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:38:11.263837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.741272+0000","last_change":"2023-10-07T07:57:50.740752+0000","last_active":"2023-10-07T07:57:50.741272+0000","last_peered":"2023-10-07T07:57:50.741272+0000","last_clean":"2023-10-07T07:57:50.741272+0000","last_became_active":"2023-10-07T07:57:50.740464+0000","last_became_peered":"2023-10-07T07:57:50.740464+0000","last_unstale":"2023-10-07T07:57:50.741272+0000","last_undegraded":"2023-10-07T07:57:50.741272+0000","last_fullsized":"2023-10-07T07:57:50.741272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:07:14.734045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.733824+0000","last_change":"2023-10-07T07:57:50.729375+0000","last_active":"2023-10-07T07:57:50.733824+0000","last_peered":"2023-10-07T07:57:50.733824+0000","last_clean":"2023-10-07T07:57:50.733824+0000","last_became_active":"2023-10-07T07:57:50.729038+0000","last_became_peered":"2023-10-07T07:57:50.729038+0000","last_unstale":"2023-10-07T07:57:50.733824+0000","last_undegraded":"2023-10-07T07:57:50.733824+0000","last_fullsized":"2023-10-07T07:57:50.733824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:04.194667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.724924+0000","last_change":"2023-10-07T07:57:50.718717+0000","last_active":"2023-10-07T07:57:50.724924+0000","last_peered":"2023-10-07T07:57:50.724924+0000","last_clean":"2023-10-07T07:57:50.724924+0000","last_became_active":"2023-10-07T07:57:50.718457+0000","last_became_peered":"2023-10-07T07:57:50.718457+0000","last_unstale":"2023-10-07T07:57:50.724924+0000","last_undegraded":"2023-10-07T07:57:50.724924+0000","last_fullsized":"2023-10-07T07:57:50.724924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:53:57.122070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-07T07:57:50.735843+0000","last_change":"2023-10-07T07:57:50.734975+0000","last_active":"2023-10-07T07:57:50.735843+0000","last_peered":"2023-10-07T07:57:50.735843+0000","last_clean":"2023-10-07T07:57:50.735843+0000","last_became_active":"2023-10-07T07:57:50.734441+0000","last_became_peered":"2023-10-07T07:57:50.734441+0000","last_unstale":"2023-10-07T07:57:50.735843+0000","last_undegraded":"2023-10-07T07:57:50.735843+0000","last_fullsized":"2023-10-07T07:57:50.735843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:49.693292+0000","last_clean_scrub_stamp":"2023-10-07T07:57:49.693292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:12.486921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":665751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.9909999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.982}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":75867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":77493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298088,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073752,"statfs":{"total":96636764160,"available":96331522048,"internally_reserved":0,"allocated":819200,"data_stored":665773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2970000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5920000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T07:58:00.217 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-07T07:58:00.218 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-07T07:58:00.218 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-07T07:58:00.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-07T07:58:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.387+0000 7f5dc50c3700 1 Processor -- start 2023-10-07T07:58:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.387+0000 7f5dc50c3700 1 -- start start 2023-10-07T07:58:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dc50c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5dc0158050 0x7f5dc01548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dc50c3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc00c97a0 con 0x7f5dc0067240 2023-10-07T07:58:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dc50c3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc0158850 con 0x7f5dc006eef0 2023-10-07T07:58:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dc50c3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc0158570 con 0x7f5dc0158050 2023-10-07T07:58:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dbe59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5dc0158050 0x7f5dc01548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dbed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5dc0067240 0x7f5dc0157940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53040/0 (socket says 172.21.15.7:53040) 2023-10-07T07:58:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dbed9d700 1 -- 172.21.15.7:0/179905778 learned_addr learned my addr 172.21.15.7:0/179905778 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dbe59c700 1 -- 172.21.15.7:0/179905778 >> v1:172.21.15.172:6789/0 conn(0x7f5dc006eef0 legacy=0x7f5dc00cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dbe59c700 1 -- 172.21.15.7:0/179905778 >> v1:172.21.15.7:6789/0 conn(0x7f5dc0067240 legacy=0x7f5dc0157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dbdd9b700 1 -- 172.21.15.7:0/179905778 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746299448 0 0) 0x7f5dc0158850 con 0x7f5dc006eef0 2023-10-07T07:58:00.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dbe59c700 1 -- 172.21.15.7:0/179905778 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc006e410 con 0x7f5dc0158050 2023-10-07T07:58:00.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.388+0000 7f5dbe59c700 1 --2- 172.21.15.7:0/179905778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0158050 0x7f5dc01548e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009910 tx=0x7f5db40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b486b1911e7bff server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:00.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dbdd9b700 1 -- 172.21.15.7:0/179905778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4009eb0 con 0x7f5dc0158050 2023-10-07T07:58:00.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dbdd9b700 1 -- 172.21.15.7:0/179905778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db400b760 con 0x7f5dc0158050 2023-10-07T07:58:00.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dbdd9b700 1 -- 172.21.15.7:0/179905778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400bb00 con 0x7f5dc0158050 2023-10-07T07:58:00.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dc50c3700 1 -- 172.21.15.7:0/179905778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0158050 msgr2=0x7f5dc01548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dc50c3700 1 --2- 172.21.15.7:0/179905778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0158050 0x7f5dc01548e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009910 tx=0x7f5db40091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dc50c3700 1 -- 172.21.15.7:0/179905778 shutdown_connections 2023-10-07T07:58:00.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dc50c3700 1 --2- 172.21.15.7:0/179905778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0158050 0x7f5dc01548e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dc50c3700 1 -- 172.21.15.7:0/179905778 >> 172.21.15.7:0/179905778 conn(0x7f5dc0068100 msgr2=0x7f5dc0068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:00.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dc50c3700 1 -- 172.21.15.7:0/179905778 shutdown_connections 2023-10-07T07:58:00.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.389+0000 7f5dc50c3700 1 -- 172.21.15.7:0/179905778 wait complete. 2023-10-07T07:58:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.391+0000 7f5dc50c3700 1 Processor -- start 2023-10-07T07:58:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.391+0000 7f5dc50c3700 1 -- start start 2023-10-07T07:58:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.391+0000 7f5dc50c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0067240 0x7f5dc015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.391+0000 7f5dc50c3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dc006eef0 0x7f5dc00c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.391+0000 7f5dc50c3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5dc0158050 0x7f5dc00c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.391+0000 7f5dc50c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc016be00 con 0x7f5dc0067240 2023-10-07T07:58:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.391+0000 7f5dc50c3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc0168d00 con 0x7f5dc0158050 2023-10-07T07:58:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.391+0000 7f5dc50c3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc016b4d0 con 0x7f5dc006eef0 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.391+0000 7f5dbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0067240 0x7f5dc015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0067240 0x7f5dc015c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47906/0 (socket says 172.21.15.7:47906) 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbed9d700 1 -- 172.21.15.7:0/3805262883 learned_addr learned my addr 172.21.15.7:0/3805262883 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbf59e700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5dc0158050 0x7f5dc00c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbe59c700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dc006eef0 0x7f5dc00c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbed9d700 1 -- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dc006eef0 msgr2=0x7f5dc00c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbed9d700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dc006eef0 0x7f5dc00c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbed9d700 1 -- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5dc0158050 msgr2=0x7f5dc00c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbed9d700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5dc0158050 0x7f5dc00c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbed9d700 1 -- 172.21.15.7:0/3805262883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4009420 con 0x7f5dc0067240 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbe59c700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dc006eef0 0x7f5dc00c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dbed9d700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0067240 0x7f5dc015c2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5db000c790 tx=0x7f5db0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5da6ffd700 1 -- 172.21.15.7:0/3805262883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0005b30 con 0x7f5dc0067240 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5da6ffd700 1 -- 172.21.15.7:0/3805262883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0005c90 con 0x7f5dc0067240 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc016b7b0 con 0x7f5dc0067240 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5da6ffd700 1 -- 172.21.15.7:0/3805262883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0007340 con 0x7f5dc0067240 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.392+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc015c800 con 0x7f5dc0067240 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.393+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc006ce00 con 0x7f5dc0067240 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.394+0000 7f5da6ffd700 1 -- 172.21.15.7:0/3805262883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5db001b400 con 0x7f5dc0067240 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.394+0000 7f5da6ffd700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5dac066f30 0x7f5dac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.394+0000 7f5da6ffd700 1 -- 172.21.15.7:0/3805262883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5db0083ff0 con 0x7f5dc0067240 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.394+0000 7f5dbe59c700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5dac066f30 0x7f5dac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.395+0000 7f5dbe59c700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5dac066f30 0x7f5dac0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5db40093f0 tx=0x7f5db4016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:00.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.399+0000 7f5da6ffd700 1 -- 172.21.15.7:0/3805262883 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5db00512e0 con 0x7f5dc0067240 2023-10-07T07:58:00.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.649+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5dc00c7c20 con 0x7f5dc0067240 2023-10-07T07:58:00.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.649+0000 7f5da6ffd700 1 -- 172.21.15.7:0/3805262883 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5db0054900 con 0x7f5dc0067240 2023-10-07T07:58:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:00.650 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-07T07:58:00.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.651+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5dac066f30 msgr2=0x7f5dac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.651+0000 7f5dc50c3700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5dac066f30 0x7f5dac0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5db40093f0 tx=0x7f5db4016000 comp rx=0 tx=0).stop 2023-10-07T07:58:00.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.651+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0067240 msgr2=0x7f5dc015c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.651+0000 7f5dc50c3700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0067240 0x7f5dc015c2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5db000c790 tx=0x7f5db0007ef0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.652+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 shutdown_connections 2023-10-07T07:58:00.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.652+0000 7f5dc50c3700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5dac066f30 0x7f5dac0693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.652+0000 7f5dc50c3700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5dc0158050 0x7f5dc00c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.652+0000 7f5dc50c3700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc0067240 0x7f5dc015c2c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.652+0000 7f5dc50c3700 1 --2- 172.21.15.7:0/3805262883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5dc006eef0 0x7f5dc00c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.652+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 >> 172.21.15.7:0/3805262883 conn(0x7f5dc0068100 msgr2=0x7f5dc00ac7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:00.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.652+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 shutdown_connections 2023-10-07T07:58:00.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.652+0000 7f5dc50c3700 1 -- 172.21.15.7:0/3805262883 wait complete. 2023-10-07T07:58:00.665 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-07T07:58:00.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:00.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.797+0000 7f24f84e4700 1 Processor -- start 2023-10-07T07:58:00.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.797+0000 7f24f84e4700 1 -- start start 2023-10-07T07:58:00.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.798+0000 7f24f84e4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24f01592c0 0x7f24f006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.798+0000 7f24f84e4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f006bb70 con 0x7f24f00cb290 2023-10-07T07:58:00.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.798+0000 7f24f84e4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f00cda10 con 0x7f24f0159690 2023-10-07T07:58:00.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.798+0000 7f24f84e4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f006af60 con 0x7f24f01592c0 2023-10-07T07:58:00.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.798+0000 7f24f6280700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24f01592c0 0x7f24f006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.798+0000 7f24f6a81700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f24f00cb290 0x7f24f00a9a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53056/0 (socket says 172.21.15.7:53056) 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.798+0000 7f24f6a81700 1 -- 172.21.15.7:0/579522562 learned_addr learned my addr 172.21.15.7:0/579522562 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.799+0000 7f24f6280700 1 -- 172.21.15.7:0/579522562 >> v1:172.21.15.172:6789/0 conn(0x7f24f0159690 legacy=0x7f24f00cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.799+0000 7f24f6280700 1 -- 172.21.15.7:0/579522562 >> v1:172.21.15.7:6789/0 conn(0x7f24f00cb290 legacy=0x7f24f00a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.799+0000 7f24f6280700 1 -- 172.21.15.7:0/579522562 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f006b240 con 0x7f24f01592c0 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.799+0000 7f24f527e700 1 -- 172.21.15.7:0/579522562 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037893737 0 0) 0x7f24f006bb70 con 0x7f24f00cb290 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.799+0000 7f24f6280700 1 --2- 172.21.15.7:0/579522562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01592c0 0x7f24f006a3f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0098e0 tx=0x7f24ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58188f9e88990047 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.799+0000 7f24f527e700 1 -- 172.21.15.7:0/579522562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec009e80 con 0x7f24f01592c0 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.799+0000 7f24f527e700 1 -- 172.21.15.7:0/579522562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec00b860 con 0x7f24f01592c0 2023-10-07T07:58:00.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.799+0000 7f24f527e700 1 -- 172.21.15.7:0/579522562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00bbe0 con 0x7f24f01592c0 2023-10-07T07:58:00.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.800+0000 7f24f84e4700 1 -- 172.21.15.7:0/579522562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01592c0 msgr2=0x7f24f006a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.800+0000 7f24f84e4700 1 --2- 172.21.15.7:0/579522562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01592c0 0x7f24f006a3f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0098e0 tx=0x7f24ec0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.800+0000 7f24f84e4700 1 -- 172.21.15.7:0/579522562 shutdown_connections 2023-10-07T07:58:00.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.800+0000 7f24f84e4700 1 --2- 172.21.15.7:0/579522562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01592c0 0x7f24f006a3f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.800+0000 7f24f84e4700 1 -- 172.21.15.7:0/579522562 >> 172.21.15.7:0/579522562 conn(0x7f24f0068bd0 msgr2=0x7f24f00ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:00.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.800+0000 7f24f84e4700 1 -- 172.21.15.7:0/579522562 shutdown_connections 2023-10-07T07:58:00.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.800+0000 7f24f84e4700 1 -- 172.21.15.7:0/579522562 wait complete. 2023-10-07T07:58:00.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.801+0000 7f24f84e4700 1 Processor -- start 2023-10-07T07:58:00.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.801+0000 7f24f84e4700 1 -- start start 2023-10-07T07:58:00.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f84e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f00cb290 0x7f24f00c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f84e4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24f01592c0 0x7f24f015b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f84e4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24f0159690 0x7f24f00c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f84e4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015c790 con 0x7f24f00cb290 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f84e4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015bb80 con 0x7f24f01592c0 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f84e4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015be60 con 0x7f24f0159690 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f6280700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f00cb290 0x7f24f00c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f6280700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f00cb290 0x7f24f00c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47924/0 (socket says 172.21.15.7:47924) 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f6280700 1 -- 172.21.15.7:0/3534047682 learned_addr learned my addr 172.21.15.7:0/3534047682 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f5a7f700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24f01592c0 0x7f24f015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f6a81700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24f0159690 0x7f24f00c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f6280700 1 -- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24f0159690 msgr2=0x7f24f00c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f6280700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24f0159690 0x7f24f00c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f6280700 1 -- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24f01592c0 msgr2=0x7f24f015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f6280700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24f01592c0 0x7f24f015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f6280700 1 -- 172.21.15.7:0/3534047682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec009420 con 0x7f24f00cb290 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.802+0000 7f24f5a7f700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24f01592c0 0x7f24f015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.803+0000 7f24f6280700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f00cb290 0x7f24f00c86b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24f0069020 tx=0x7f24ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.803+0000 7f24e27fc700 1 -- 172.21.15.7:0/3534047682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec01f070 con 0x7f24f00cb290 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.803+0000 7f24e27fc700 1 -- 172.21.15.7:0/3534047682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec00b9d0 con 0x7f24f00cb290 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.803+0000 7f24e27fc700 1 -- 172.21.15.7:0/3534047682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec025670 con 0x7f24f00cb290 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.803+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f015c140 con 0x7f24f00cb290 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.803+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f0169390 con 0x7f24f00cb290 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.804+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f00716c0 con 0x7f24f00cb290 2023-10-07T07:58:00.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.804+0000 7f24e27fc700 1 -- 172.21.15.7:0/3534047682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24ec023070 con 0x7f24f00cb290 2023-10-07T07:58:00.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.804+0000 7f24e27fc700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24d4066f30 0x7f24d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:00.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.805+0000 7f24e27fc700 1 -- 172.21.15.7:0/3534047682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f24ec085bc0 con 0x7f24f00cb290 2023-10-07T07:58:00.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.805+0000 7f24f5a7f700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24d4066f30 0x7f24d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:00.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.805+0000 7f24f5a7f700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24d4066f30 0x7f24d40693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f24f016c620 tx=0x7f24e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:00.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:00.809+0000 7f24e27fc700 1 -- 172.21.15.7:0/3534047682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24ec052e30 con 0x7f24f00cb290 2023-10-07T07:58:01.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.015+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24f0158b60 con 0x7f24f00cb290 2023-10-07T07:58:01.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.016+0000 7f24e27fc700 1 -- 172.21.15.7:0/3534047682 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f24ec02a090 con 0x7f24f00cb290 2023-10-07T07:58:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:01.016 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24d4066f30 msgr2=0x7f24d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24d4066f30 0x7f24d40693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f24f016c620 tx=0x7f24e4009040 comp rx=0 tx=0).stop 2023-10-07T07:58:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f00cb290 msgr2=0x7f24f00c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f00cb290 0x7f24f00c86b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24f0069020 tx=0x7f24ec017000 comp rx=0 tx=0).stop 2023-10-07T07:58:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 shutdown_connections 2023-10-07T07:58:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24d4066f30 0x7f24d40693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24f0159690 0x7f24f00c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f00cb290 0x7f24f00c86b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 --2- 172.21.15.7:0/3534047682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24f01592c0 0x7f24f015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 >> 172.21.15.7:0/3534047682 conn(0x7f24f0068bd0 msgr2=0x7f24f00673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:01.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 shutdown_connections 2023-10-07T07:58:01.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.018+0000 7f24f84e4700 1 -- 172.21.15.7:0/3534047682 wait complete. 2023-10-07T07:58:01.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T07:57:48.285077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T07:57:49.111979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T07:58:01.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:58:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.167+0000 7feca4128700 1 Processor -- start 2023-10-07T07:58:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.167+0000 7feca4128700 1 -- start start 2023-10-07T07:58:01.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.168+0000 7feca4128700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fec9c157b20 0x7fec9c157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.168+0000 7feca4128700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c15e7a0 0x7fec9c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.168+0000 7feca4128700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c161380 con 0x7fec9c0affc0 2023-10-07T07:58:01.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.168+0000 7feca4128700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c0b0440 con 0x7fec9c157b20 2023-10-07T07:58:01.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.168+0000 7feca4128700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c161660 con 0x7fec9c15e7a0 2023-10-07T07:58:01.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.168+0000 7feca1ec4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fec9c157b20 0x7fec9c157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca1ec4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fec9c157b20 0x7fec9c157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47932/0 (socket says 172.21.15.7:47932) 2023-10-07T07:58:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca1ec4700 1 -- 172.21.15.7:0/1149671150 learned_addr learned my addr 172.21.15.7:0/1149671150 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca16c3700 1 --2- 172.21.15.7:0/1149671150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c15e7a0 0x7fec9c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca1ec4700 1 -- 172.21.15.7:0/1149671150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c15e7a0 msgr2=0x7fec9c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca1ec4700 1 --2- 172.21.15.7:0/1149671150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c15e7a0 0x7fec9c0afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca1ec4700 1 -- 172.21.15.7:0/1149671150 >> v1:172.21.15.172:6789/0 conn(0x7fec9c0affc0 legacy=0x7fec9c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca1ec4700 1 -- 172.21.15.7:0/1149671150 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c161940 con 0x7fec9c157b20 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca1ec4700 1 --2- 172.21.15.7:0/1149671150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c157b20 0x7fec9c157ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fec980099b0 tx=0x7fec98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45c26cfaa8322db server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca0ec2700 1 -- 172.21.15.7:0/1149671150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98009ea0 con 0x7fec9c157b20 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca0ec2700 1 -- 172.21.15.7:0/1149671150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9800b860 con 0x7fec9c157b20 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.169+0000 7feca0ec2700 1 -- 172.21.15.7:0/1149671150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9800bc00 con 0x7fec9c157b20 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.170+0000 7feca4128700 1 -- 172.21.15.7:0/1149671150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c157b20 msgr2=0x7fec9c157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.170+0000 7feca4128700 1 --2- 172.21.15.7:0/1149671150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c157b20 0x7fec9c157ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fec980099b0 tx=0x7fec98009250 comp rx=0 tx=0).stop 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.170+0000 7feca4128700 1 -- 172.21.15.7:0/1149671150 shutdown_connections 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.170+0000 7feca4128700 1 --2- 172.21.15.7:0/1149671150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c157b20 0x7fec9c157ef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.170+0000 7feca4128700 1 --2- 172.21.15.7:0/1149671150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c15e7a0 0x7fec9c0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.170+0000 7feca4128700 1 -- 172.21.15.7:0/1149671150 >> 172.21.15.7:0/1149671150 conn(0x7fec9c06d350 msgr2=0x7fec9c06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:01.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.170+0000 7feca4128700 1 -- 172.21.15.7:0/1149671150 shutdown_connections 2023-10-07T07:58:01.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.170+0000 7feca4128700 1 -- 172.21.15.7:0/1149671150 wait complete. 2023-10-07T07:58:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.171+0000 7feca4128700 1 Processor -- start 2023-10-07T07:58:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.171+0000 7feca4128700 1 -- start start 2023-10-07T07:58:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca4128700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c0affc0 0x7fec9c150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca4128700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c157b20 0x7fec9c150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca4128700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15e7a0 0x7fec9c170980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca4128700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0af370 con 0x7fec9c15e7a0 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca4128700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0ae760 con 0x7fec9c157b20 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca4128700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0aea40 con 0x7fec9c0affc0 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca26c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15e7a0 0x7fec9c170980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca26c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15e7a0 0x7fec9c170980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47946/0 (socket says 172.21.15.7:47946) 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca26c5700 1 -- 172.21.15.7:0/3119085006 learned_addr learned my addr 172.21.15.7:0/3119085006 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca1ec4700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c0affc0 0x7fec9c150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca26c5700 1 -- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c0affc0 msgr2=0x7fec9c150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca26c5700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c0affc0 0x7fec9c150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca26c5700 1 -- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c157b20 msgr2=0x7fec9c150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca26c5700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c157b20 0x7fec9c150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.172+0000 7feca26c5700 1 -- 172.21.15.7:0/3119085006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec980094f0 con 0x7fec9c15e7a0 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.173+0000 7feca26c5700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15e7a0 0x7fec9c170980 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec9400c8e0 tx=0x7fec94001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.173+0000 7fec91ffb700 1 -- 172.21.15.7:0/3119085006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94012630 con 0x7fec9c15e7a0 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.173+0000 7fec91ffb700 1 -- 172.21.15.7:0/3119085006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec94012790 con 0x7fec9c15e7a0 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.173+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c0aed20 con 0x7fec9c15e7a0 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.173+0000 7fec91ffb700 1 -- 172.21.15.7:0/3119085006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94012b50 con 0x7fec9c15e7a0 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.173+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c0ae450 con 0x7fec9c15e7a0 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.174+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c04c3c0 con 0x7fec9c15e7a0 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.174+0000 7fec91ffb700 1 -- 172.21.15.7:0/3119085006 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec9400a3b0 con 0x7fec9c15e7a0 2023-10-07T07:58:01.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.175+0000 7fec91ffb700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec80066fb0 0x7fec80069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.175+0000 7fec91ffb700 1 -- 172.21.15.7:0/3119085006 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec94084850 con 0x7fec9c15e7a0 2023-10-07T07:58:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.175+0000 7feca1ec4700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec80066fb0 0x7fec80069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.176+0000 7feca1ec4700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec80066fb0 0x7fec80069460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fec980099b0 tx=0x7fec98017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.179+0000 7fec91ffb700 1 -- 172.21.15.7:0/3119085006 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec940554b0 con 0x7fec9c15e7a0 2023-10-07T07:58:01.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.382+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec9c161c20 con 0x7fec9c15e7a0 2023-10-07T07:58:01.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.382+0000 7fec91ffb700 1 -- 172.21.15.7:0/3119085006 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec94055040 con 0x7fec9c15e7a0 2023-10-07T07:58:01.383 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:58:01.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec80066fb0 msgr2=0x7fec80069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec80066fb0 0x7fec80069460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fec980099b0 tx=0x7fec98017000 comp rx=0 tx=0).stop 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15e7a0 msgr2=0x7fec9c170980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15e7a0 0x7fec9c170980 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec9400c8e0 tx=0x7fec94001710 comp rx=0 tx=0).stop 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 shutdown_connections 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec80066fb0 0x7fec80069460 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15e7a0 0x7fec9c170980 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec9c0affc0 0x7fec9c150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 --2- 172.21.15.7:0/3119085006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c157b20 0x7fec9c150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 >> 172.21.15.7:0/3119085006 conn(0x7fec9c06d350 msgr2=0x7fec9c155740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 shutdown_connections 2023-10-07T07:58:01.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.385+0000 7feca4128700 1 -- 172.21.15.7:0/3119085006 wait complete. 2023-10-07T07:58:01.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T07:58:01.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.533+0000 7f944e6a7700 1 Processor -- start 2023-10-07T07:58:01.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.533+0000 7f944e6a7700 1 -- start start 2023-10-07T07:58:01.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.533+0000 7f944e6a7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9448155d90 0x7f9448156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.533+0000 7f944e6a7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f94480c7a50 0x7f944806c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.533+0000 7f944e6a7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944806b6f0 con 0x7f94480c75e0 2023-10-07T07:58:01.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.533+0000 7f944e6a7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94480cbbf0 con 0x7f94480c7a50 2023-10-07T07:58:01.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.533+0000 7f944e6a7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944806bf70 con 0x7f9448155d90 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f944cc44700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f94480c7a50 0x7f944806c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f94477fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f94480c75e0 0x7f944806e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53060/0 (socket says 172.21.15.7:53060) 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f94477fe700 1 -- 172.21.15.7:0/1833938257 learned_addr learned my addr 172.21.15.7:0/1833938257 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f944cc44700 1 -- 172.21.15.7:0/1833938257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9448155d90 msgr2=0x7f9448156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f944cc44700 1 --2- 172.21.15.7:0/1833938257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9448155d90 0x7f9448156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f944cc44700 1 -- 172.21.15.7:0/1833938257 >> v1:172.21.15.7:6789/0 conn(0x7f94480c75e0 legacy=0x7f944806e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f944cc44700 1 -- 172.21.15.7:0/1833938257 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94480cb910 con 0x7f94480c7a50 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f944cc44700 1 --2- 172.21.15.7:0/1833938257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 0x7f944806c400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9440009e60 tx=0x7f9440009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858182afdd547a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f9446ffd700 1 -- 172.21.15.7:0/1833938257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000bc30 con 0x7f94480c7a50 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f9446ffd700 1 -- 172.21.15.7:0/1833938257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944000bd90 con 0x7f94480c7a50 2023-10-07T07:58:01.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.534+0000 7f9446ffd700 1 -- 172.21.15.7:0/1833938257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400116b0 con 0x7f94480c7a50 2023-10-07T07:58:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.535+0000 7f944e6a7700 1 -- 172.21.15.7:0/1833938257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 msgr2=0x7f944806c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.535+0000 7f944e6a7700 1 --2- 172.21.15.7:0/1833938257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 0x7f944806c400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9440009e60 tx=0x7f9440009780 comp rx=0 tx=0).stop 2023-10-07T07:58:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.535+0000 7f944e6a7700 1 -- 172.21.15.7:0/1833938257 shutdown_connections 2023-10-07T07:58:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.535+0000 7f944e6a7700 1 --2- 172.21.15.7:0/1833938257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 0x7f944806c400 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.535+0000 7f944e6a7700 1 --2- 172.21.15.7:0/1833938257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9448155d90 0x7f9448156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.535+0000 7f944e6a7700 1 -- 172.21.15.7:0/1833938257 >> 172.21.15.7:0/1833938257 conn(0x7f9448068100 msgr2=0x7f9448068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.535+0000 7f944e6a7700 1 -- 172.21.15.7:0/1833938257 shutdown_connections 2023-10-07T07:58:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.535+0000 7f944e6a7700 1 -- 172.21.15.7:0/1833938257 wait complete. 2023-10-07T07:58:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.536+0000 7f944e6a7700 1 Processor -- start 2023-10-07T07:58:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.536+0000 7f944e6a7700 1 -- start start 2023-10-07T07:58:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f944e6a7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94480c75e0 0x7f944814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f944e6a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 0x7f9448151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f944e6a7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9448155d90 0x7f9448151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f944e6a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94481507f0 con 0x7f94480c7a50 2023-10-07T07:58:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f944e6a7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9448150db0 con 0x7f9448155d90 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f944e6a7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94480addb0 con 0x7f94480c75e0 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f94477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 0x7f9448151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f94477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 0x7f9448151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47970/0 (socket says 172.21.15.7:47970) 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f94477fe700 1 -- 172.21.15.7:0/3811504841 learned_addr learned my addr 172.21.15.7:0/3811504841 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f9447fff700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94480c75e0 0x7f944814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.537+0000 7f944cc44700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9448155d90 0x7f9448151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f94477fe700 1 -- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94480c75e0 msgr2=0x7f944814d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f94477fe700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94480c75e0 0x7f944814d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f94477fe700 1 -- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9448155d90 msgr2=0x7f9448151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f94477fe700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9448155d90 0x7f9448151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f94477fe700 1 -- 172.21.15.7:0/3811504841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440009a20 con 0x7f94480c7a50 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f94477fe700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 0x7f9448151240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c7f0 tx=0x7f943c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f942ffff700 1 -- 172.21.15.7:0/3811504841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c005b30 con 0x7f94480c7a50 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f942ffff700 1 -- 172.21.15.7:0/3811504841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c005cb0 con 0x7f94480c7a50 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f942ffff700 1 -- 172.21.15.7:0/3811504841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c004a80 con 0x7f94480c7a50 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94480ae090 con 0x7f94480c7a50 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.538+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944814d890 con 0x7f94480c7a50 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.539+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9448150f40 con 0x7f94480c7a50 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.539+0000 7f942ffff700 1 -- 172.21.15.7:0/3811504841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f943c0070d0 con 0x7f94480c7a50 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.540+0000 7f942ffff700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9434066f30 0x7f94340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.540+0000 7f942ffff700 1 -- 172.21.15.7:0/3811504841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f943c083800 con 0x7f94480c7a50 2023-10-07T07:58:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.540+0000 7f9447fff700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9434066f30 0x7f94340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.541+0000 7f9447fff700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9434066f30 0x7f94340693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9438009b40 tx=0x7f94380090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.544+0000 7f942ffff700 1 -- 172.21.15.7:0/3811504841 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f943c04b5a0 con 0x7f94480c7a50 2023-10-07T07:58:01.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.747+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f944804c3c0 con 0x7f94480c7a50 2023-10-07T07:58:01.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.747+0000 7f942ffff700 1 -- 172.21.15.7:0/3811504841 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f943c050a70 con 0x7f94480c7a50 2023-10-07T07:58:01.748 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-07T07:58:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.749+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9434066f30 msgr2=0x7f94340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.749+0000 7f944e6a7700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9434066f30 0x7f94340693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9438009b40 tx=0x7f94380090f0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.749+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 msgr2=0x7f9448151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.749+0000 7f944e6a7700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 0x7f9448151240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c7f0 tx=0x7f943c007f70 comp rx=0 tx=0).stop 2023-10-07T07:58:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.750+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 shutdown_connections 2023-10-07T07:58:01.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.750+0000 7f944e6a7700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9434066f30 0x7f94340693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.750+0000 7f944e6a7700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9448155d90 0x7f9448151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.750+0000 7f944e6a7700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94480c75e0 0x7f944814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.750+0000 7f944e6a7700 1 --2- 172.21.15.7:0/3811504841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94480c7a50 0x7f9448151240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.750+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 >> 172.21.15.7:0/3811504841 conn(0x7f9448068100 msgr2=0x7f944814f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:01.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.750+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 shutdown_connections 2023-10-07T07:58:01.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.750+0000 7f944e6a7700 1 -- 172.21.15.7:0/3811504841 wait complete. 2023-10-07T07:58:01.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T07:58:01.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.897+0000 7f24733b3700 1 Processor -- start 2023-10-07T07:58:01.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.898+0000 7f24733b3700 1 -- start start 2023-10-07T07:58:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f24733b3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f246c156f30 0x7f246c0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f24733b3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c0b22a0 0x7f246c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f24733b3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246c0b18d0 con 0x7f246c157300 2023-10-07T07:58:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f24733b3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246c0b0fa0 con 0x7f246c156f30 2023-10-07T07:58:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f24733b3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0b1280 con 0x7f246c0b22a0 2023-10-07T07:58:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f2471950700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f246c157300 0x7f246c0b3060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53070/0 (socket says 172.21.15.7:53070) 2023-10-07T07:58:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f2471950700 1 -- 172.21.15.7:0/2087388143 learned_addr learned my addr 172.21.15.7:0/2087388143 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f247094e700 1 --2- 172.21.15.7:0/2087388143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c0b22a0 0x7f246c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f247114f700 1 --2- 172.21.15.7:0/2087388143 >> v2:172.21.15.172:3300/0 conn(0x7f246c156f30 0x7f246c0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f2463fff700 1 -- 172.21.15.7:0/2087388143 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629540090 0 0) 0x7f246c0b18d0 con 0x7f246c157300 2023-10-07T07:58:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f2463fff700 1 -- 172.21.15.7:0/2087388143 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2458003570 con 0x7f246c157300 2023-10-07T07:58:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f247094e700 1 -- 172.21.15.7:0/2087388143 >> v2:172.21.15.172:3300/0 conn(0x7f246c156f30 msgr2=0x7f246c0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f247094e700 1 --2- 172.21.15.7:0/2087388143 >> v2:172.21.15.172:3300/0 conn(0x7f246c156f30 0x7f246c0b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f247094e700 1 -- 172.21.15.7:0/2087388143 >> v1:172.21.15.7:6789/0 conn(0x7f246c157300 legacy=0x7f246c0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f247094e700 1 -- 172.21.15.7:0/2087388143 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c0b1560 con 0x7f246c0b22a0 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f247114f700 1 --2- 172.21.15.7:0/2087388143 >> v2:172.21.15.172:3300/0 conn(0x7f246c156f30 0x7f246c0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.899+0000 7f247094e700 1 --2- 172.21.15.7:0/2087388143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c0b22a0 0x7f246c0afa80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f24680098e0 tx=0x7f24680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a0fdb0ba5ca4f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f2463fff700 1 -- 172.21.15.7:0/2087388143 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468011630 con 0x7f246c0b22a0 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f2463fff700 1 -- 172.21.15.7:0/2087388143 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2468011790 con 0x7f246c0b22a0 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f2463fff700 1 -- 172.21.15.7:0/2087388143 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468011b30 con 0x7f246c0b22a0 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f24733b3700 1 -- 172.21.15.7:0/2087388143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c0b22a0 msgr2=0x7f246c0afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f24733b3700 1 --2- 172.21.15.7:0/2087388143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c0b22a0 0x7f246c0afa80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f24680098e0 tx=0x7f24680091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f24733b3700 1 -- 172.21.15.7:0/2087388143 shutdown_connections 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f24733b3700 1 --2- 172.21.15.7:0/2087388143 >> v2:172.21.15.172:3300/0 conn(0x7f246c156f30 0x7f246c0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f24733b3700 1 --2- 172.21.15.7:0/2087388143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c0b22a0 0x7f246c0afa80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f24733b3700 1 -- 172.21.15.7:0/2087388143 >> 172.21.15.7:0/2087388143 conn(0x7f246c06d350 msgr2=0x7f246c06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f24733b3700 1 -- 172.21.15.7:0/2087388143 shutdown_connections 2023-10-07T07:58:01.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.900+0000 7f24733b3700 1 -- 172.21.15.7:0/2087388143 wait complete. 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.902+0000 7f24733b3700 1 Processor -- start 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.902+0000 7f24733b3700 1 -- start start 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.902+0000 7f24733b3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f246c0b22a0 0x7f246c068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.902+0000 7f24733b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f246c156f30 0x7f246c14df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.902+0000 7f24733b3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c157300 0x7f246c0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.902+0000 7f24733b3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246c069000 con 0x7f246c156f30 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f24733b3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0c7590 con 0x7f246c0b22a0 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f24733b3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0c9990 con 0x7f246c157300 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247094e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f246c156f30 0x7f246c14df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247094e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f246c156f30 0x7f246c14df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47982/0 (socket says 172.21.15.7:47982) 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247094e700 1 -- 172.21.15.7:0/3486777672 learned_addr learned my addr 172.21.15.7:0/3486777672 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247114f700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f246c0b22a0 0x7f246c068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f2471950700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c157300 0x7f246c0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247094e700 1 -- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c157300 msgr2=0x7f246c0add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247094e700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c157300 0x7f246c0add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247094e700 1 -- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f246c0b22a0 msgr2=0x7f246c068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247094e700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f246c0b22a0 0x7f246c068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247094e700 1 -- 172.21.15.7:0/3486777672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245c009420 con 0x7f246c156f30 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f2471950700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c157300 0x7f246c0add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247114f700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f246c0b22a0 0x7f246c068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f247094e700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f246c156f30 0x7f246c14df00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2468009ed0 tx=0x7f2468017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f24617fa700 1 -- 172.21.15.7:0/3486777672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468011b10 con 0x7f246c156f30 2023-10-07T07:58:01.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f24617fa700 1 -- 172.21.15.7:0/3486777672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2468006af0 con 0x7f246c156f30 2023-10-07T07:58:01.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2468009420 con 0x7f246c156f30 2023-10-07T07:58:01.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f24617fa700 1 -- 172.21.15.7:0/3486777672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468025640 con 0x7f246c156f30 2023-10-07T07:58:01.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.903+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246c0c9060 con 0x7f246c156f30 2023-10-07T07:58:01.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.904+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246c0c9b40 con 0x7f246c156f30 2023-10-07T07:58:01.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.905+0000 7f24617fa700 1 -- 172.21.15.7:0/3486777672 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2468025a70 con 0x7f246c156f30 2023-10-07T07:58:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.905+0000 7f24617fa700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2450066f30 0x7f24500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.906+0000 7f24617fa700 1 -- 172.21.15.7:0/3486777672 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2468085cd0 con 0x7f246c156f30 2023-10-07T07:58:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.906+0000 7f247114f700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2450066f30 0x7f24500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.906+0000 7f247114f700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2450066f30 0x7f24500693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f245c0093f0 tx=0x7f245c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:01.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:01.910+0000 7f24617fa700 1 -- 172.21.15.7:0/3486777672 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f246c0c9b40 con 0x7f246c156f30 2023-10-07T07:58:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.113+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f246c04c420 con 0x7f246c156f30 2023-10-07T07:58:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.113+0000 7f24617fa700 1 -- 172.21.15.7:0/3486777672 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24680150a0 con 0x7f246c156f30 2023-10-07T07:58:02.113 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-07T07:58:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.115+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2450066f30 msgr2=0x7f24500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.115+0000 7f24733b3700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2450066f30 0x7f24500693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f245c0093f0 tx=0x7f245c008040 comp rx=0 tx=0).stop 2023-10-07T07:58:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.115+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f246c156f30 msgr2=0x7f246c14df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.115+0000 7f24733b3700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f246c156f30 0x7f246c14df00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2468009ed0 tx=0x7f2468017000 comp rx=0 tx=0).stop 2023-10-07T07:58:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.116+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 shutdown_connections 2023-10-07T07:58:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.116+0000 7f24733b3700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2450066f30 0x7f24500693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.116+0000 7f24733b3700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f246c157300 0x7f246c0add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.116+0000 7f24733b3700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f246c0b22a0 0x7f246c068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.116+0000 7f24733b3700 1 --2- 172.21.15.7:0/3486777672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f246c156f30 0x7f246c14df00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.116+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 >> 172.21.15.7:0/3486777672 conn(0x7f246c06d350 msgr2=0x7f246c157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.116+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 shutdown_connections 2023-10-07T07:58:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.116+0000 7f24733b3700 1 -- 172.21.15.7:0/3486777672 wait complete. 2023-10-07T07:58:02.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:58:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.262+0000 7f824393c700 1 Processor -- start 2023-10-07T07:58:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.263+0000 7f824393c700 1 -- start start 2023-10-07T07:58:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.263+0000 7f824393c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f823c0c9af0 0x7f823c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f824393c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c063230 0x7f823c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f824393c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f823c0c9ec0 0x7f823c06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f824393c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c0b03a0 con 0x7f823c0c9ec0 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f824393c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c06e590 con 0x7f823c0c9af0 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f824393c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c15a330 con 0x7f823c063230 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8241ed9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f823c0c9ec0 0x7f823c06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8241ed9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f823c0c9ec0 0x7f823c06e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47994/0 (socket says 172.21.15.7:47994) 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8241ed9700 1 -- 172.21.15.7:0/3531107936 learned_addr learned my addr 172.21.15.7:0/3531107936 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f82416d8700 1 --2- 172.21.15.7:0/3531107936 >> v2:172.21.15.172:3300/0 conn(0x7f823c0c9af0 0x7f823c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8240ed7700 1 --2- 172.21.15.7:0/3531107936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c063230 0x7f823c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8241ed9700 1 -- 172.21.15.7:0/3531107936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c063230 msgr2=0x7f823c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8241ed9700 1 --2- 172.21.15.7:0/3531107936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c063230 0x7f823c06db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8241ed9700 1 -- 172.21.15.7:0/3531107936 >> v2:172.21.15.172:3300/0 conn(0x7f823c0c9af0 msgr2=0x7f823c15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8241ed9700 1 --2- 172.21.15.7:0/3531107936 >> v2:172.21.15.172:3300/0 conn(0x7f823c0c9af0 0x7f823c15d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8241ed9700 1 -- 172.21.15.7:0/3531107936 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c15a610 con 0x7f823c0c9ec0 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8240ed7700 1 --2- 172.21.15.7:0/3531107936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c063230 0x7f823c06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8241ed9700 1 --2- 172.21.15.7:0/3531107936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0c9ec0 0x7f823c06e050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f823c0ca400 tx=0x7f8234009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab89391ccde6d4e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.264+0000 7f8233fff700 1 -- 172.21.15.7:0/3531107936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400e070 con 0x7f823c0c9ec0 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f8233fff700 1 -- 172.21.15.7:0/3531107936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823400ba20 con 0x7f823c0c9ec0 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f8233fff700 1 -- 172.21.15.7:0/3531107936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82340092e0 con 0x7f823c0c9ec0 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f824393c700 1 -- 172.21.15.7:0/3531107936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0c9ec0 msgr2=0x7f823c06e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f824393c700 1 --2- 172.21.15.7:0/3531107936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0c9ec0 0x7f823c06e050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f823c0ca400 tx=0x7f8234009840 comp rx=0 tx=0).stop 2023-10-07T07:58:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f824393c700 1 -- 172.21.15.7:0/3531107936 shutdown_connections 2023-10-07T07:58:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f824393c700 1 --2- 172.21.15.7:0/3531107936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0c9ec0 0x7f823c06e050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f824393c700 1 --2- 172.21.15.7:0/3531107936 >> v2:172.21.15.172:3300/0 conn(0x7f823c0c9af0 0x7f823c15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f824393c700 1 --2- 172.21.15.7:0/3531107936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c063230 0x7f823c06db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f824393c700 1 -- 172.21.15.7:0/3531107936 >> 172.21.15.7:0/3531107936 conn(0x7f823c0cd1a0 msgr2=0x7f823c0c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f824393c700 1 -- 172.21.15.7:0/3531107936 shutdown_connections 2023-10-07T07:58:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.265+0000 7f824393c700 1 -- 172.21.15.7:0/3531107936 wait complete. 2023-10-07T07:58:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.267+0000 7f824393c700 1 Processor -- start 2023-10-07T07:58:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.267+0000 7f824393c700 1 -- start start 2023-10-07T07:58:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.267+0000 7f824393c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c063230 0x7f823c147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f824393c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c0c9af0 0x7f823c1694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f824393c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c0c9ec0 0x7f823c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f824393c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c156f70 con 0x7f823c063230 2023-10-07T07:58:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f824393c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c15b3d0 con 0x7f823c0c9ec0 2023-10-07T07:58:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f824393c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0c8620 con 0x7f823c0c9af0 2023-10-07T07:58:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82416d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c063230 0x7f823c147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82416d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c063230 0x7f823c147160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47996/0 (socket says 172.21.15.7:47996) 2023-10-07T07:58:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82416d8700 1 -- 172.21.15.7:0/2904770005 learned_addr learned my addr 172.21.15.7:0/2904770005 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f8241ed9700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c0c9ec0 0x7f823c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f8240ed7700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c0c9af0 0x7f823c1694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82416d8700 1 -- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c0c9af0 msgr2=0x7f823c1694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82416d8700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c0c9af0 0x7f823c1694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82416d8700 1 -- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c0c9ec0 msgr2=0x7f823c157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82416d8700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c0c9ec0 0x7f823c157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82416d8700 1 -- 172.21.15.7:0/2904770005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8234009ae0 con 0x7f823c063230 2023-10-07T07:58:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82416d8700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c063230 0x7f823c147160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f823800ec80 tx=0x7f82380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82317fa700 1 -- 172.21.15.7:0/2904770005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823800c850 con 0x7f823c063230 2023-10-07T07:58:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82317fa700 1 -- 172.21.15.7:0/2904770005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823800c9b0 con 0x7f823c063230 2023-10-07T07:58:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c1476a0 con 0x7f823c063230 2023-10-07T07:58:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f82317fa700 1 -- 172.21.15.7:0/2904770005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823800cd50 con 0x7f823c063230 2023-10-07T07:58:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.268+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c0c8240 con 0x7f823c063230 2023-10-07T07:58:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.269+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c0c8490 con 0x7f823c063230 2023-10-07T07:58:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.270+0000 7f82317fa700 1 -- 172.21.15.7:0/2904770005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8238013680 con 0x7f823c063230 2023-10-07T07:58:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.270+0000 7f82317fa700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8220066f30 0x7f82200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.271+0000 7f82317fa700 1 -- 172.21.15.7:0/2904770005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8238084800 con 0x7f823c063230 2023-10-07T07:58:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.271+0000 7f8240ed7700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8220066f30 0x7f82200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.271+0000 7f8240ed7700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8220066f30 0x7f82200693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f823c154f90 tx=0x7f822c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.275+0000 7f82317fa700 1 -- 172.21.15.7:0/2904770005 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82380519d0 con 0x7f823c063230 2023-10-07T07:58:02.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.520+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f823c157250 con 0x7f823c063230 2023-10-07T07:58:02.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.520+0000 7f82317fa700 1 -- 172.21.15.7:0/2904770005 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f8238054ff0 con 0x7f823c063230 2023-10-07T07:58:02.521 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:02.521 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2318752844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2318752844},{"type":"v1","addr":"172.21.15.172:6836","nonce":2318752844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1231430544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1231430544},{"type":"v1","addr":"172.21.15.172:6837","nonce":1231430544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1421504798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1421504798},{"type":"v1","addr":"172.21.15.7:6833","nonce":1421504798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:57:50.139330+0000","modified":"2023-10-07T07:57:53.727131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T07:58:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.522+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8220066f30 msgr2=0x7f82200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.522+0000 7f824393c700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8220066f30 0x7f82200693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f823c154f90 tx=0x7f822c008040 comp rx=0 tx=0).stop 2023-10-07T07:58:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.522+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c063230 msgr2=0x7f823c147160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.522+0000 7f824393c700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c063230 0x7f823c147160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f823800ec80 tx=0x7f82380091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.523+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 shutdown_connections 2023-10-07T07:58:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.523+0000 7f824393c700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8220066f30 0x7f82200693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.523+0000 7f824393c700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c0c9ec0 0x7f823c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.523+0000 7f824393c700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c063230 0x7f823c147160 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.523+0000 7f824393c700 1 --2- 172.21.15.7:0/2904770005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f823c0c9af0 0x7f823c1694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.523+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 >> 172.21.15.7:0/2904770005 conn(0x7f823c0cd1a0 msgr2=0x7f823c066ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:02.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.523+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 shutdown_connections 2023-10-07T07:58:02.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.523+0000 7f824393c700 1 -- 172.21.15.7:0/2904770005 wait complete. 2023-10-07T07:58:02.524 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2023-10-07T07:58:02.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:02.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.668+0000 7f83b11e7700 1 Processor -- start 2023-10-07T07:58:02.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.668+0000 7f83b11e7700 1 -- start start 2023-10-07T07:58:02.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83b11e7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f83ac155f20 0x7f83ac0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83b11e7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83ac06e9d0 0x7f83ac06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83b11e7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ac06b700 con 0x7f83ac1562f0 2023-10-07T07:58:02.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83b11e7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ac06aaf0 con 0x7f83ac06e9d0 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83b11e7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ac06add0 con 0x7f83ac155f20 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83ab59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83ac06e9d0 0x7f83ac06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83ab59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83ac06e9d0 0x7f83ac06a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48008/0 (socket says 172.21.15.7:48008) 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83ab59e700 1 -- 172.21.15.7:0/2377180722 learned_addr learned my addr 172.21.15.7:0/2377180722 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83aad9d700 1 --2- 172.21.15.7:0/2377180722 >> v2:172.21.15.172:3300/0 conn(0x7f83ac155f20 0x7f83ac0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83ab59e700 1 -- 172.21.15.7:0/2377180722 >> v2:172.21.15.172:3300/0 conn(0x7f83ac155f20 msgr2=0x7f83ac0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83ab59e700 1 --2- 172.21.15.7:0/2377180722 >> v2:172.21.15.172:3300/0 conn(0x7f83ac155f20 0x7f83ac0c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83ab59e700 1 -- 172.21.15.7:0/2377180722 >> v1:172.21.15.172:6789/0 conn(0x7f83ac1562f0 legacy=0x7f83ac06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83ab59e700 1 -- 172.21.15.7:0/2377180722 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac06b0b0 con 0x7f83ac06e9d0 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83aad9d700 1 --2- 172.21.15.7:0/2377180722 >> v2:172.21.15.172:3300/0 conn(0x7f83ac155f20 0x7f83ac0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83ab59e700 1 --2- 172.21.15.7:0/2377180722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac06e9d0 0x7f83ac06a3f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83ac070c00 tx=0x7f83a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfbfc4228ed617e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:02.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83a9d9b700 1 -- 172.21.15.7:0/2377180722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a400e070 con 0x7f83ac06e9d0 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.669+0000 7f83a9d9b700 1 -- 172.21.15.7:0/2377180722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a400ba50 con 0x7f83ac06e9d0 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.670+0000 7f83a9d9b700 1 -- 172.21.15.7:0/2377180722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a40092e0 con 0x7f83ac06e9d0 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.670+0000 7f83b11e7700 1 -- 172.21.15.7:0/2377180722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac06e9d0 msgr2=0x7f83ac06a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.670+0000 7f83b11e7700 1 --2- 172.21.15.7:0/2377180722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac06e9d0 0x7f83ac06a3f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83ac070c00 tx=0x7f83a4009840 comp rx=0 tx=0).stop 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.670+0000 7f83b11e7700 1 -- 172.21.15.7:0/2377180722 shutdown_connections 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.670+0000 7f83b11e7700 1 --2- 172.21.15.7:0/2377180722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac06e9d0 0x7f83ac06a3f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.670+0000 7f83b11e7700 1 --2- 172.21.15.7:0/2377180722 >> v2:172.21.15.172:3300/0 conn(0x7f83ac155f20 0x7f83ac0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.670+0000 7f83b11e7700 1 -- 172.21.15.7:0/2377180722 >> 172.21.15.7:0/2377180722 conn(0x7f83ac068bd0 msgr2=0x7f83ac0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.670+0000 7f83b11e7700 1 -- 172.21.15.7:0/2377180722 shutdown_connections 2023-10-07T07:58:02.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.670+0000 7f83b11e7700 1 -- 172.21.15.7:0/2377180722 wait complete. 2023-10-07T07:58:02.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.671+0000 7f83b11e7700 1 Processor -- start 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.671+0000 7f83b11e7700 1 -- start start 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83b11e7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ac06e9d0 0x7f83ac0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83b11e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac155f20 0x7f83ac14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83b11e7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ac1562f0 0x7f83ac0c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83b11e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac153240 con 0x7f83ac155f20 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83b11e7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac0b2240 con 0x7f83ac06e9d0 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83b11e7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac152350 con 0x7f83ac1562f0 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83aa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac155f20 0x7f83ac14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83aa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac155f20 0x7f83ac14f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48022/0 (socket says 172.21.15.7:48022) 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83aa59c700 1 -- 172.21.15.7:0/2305097044 learned_addr learned my addr 172.21.15.7:0/2305097044 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:02.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.672+0000 7f83aad9d700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ac06e9d0 0x7f83ac0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f83ab59e700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ac1562f0 0x7f83ac0c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f83aa59c700 1 -- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ac1562f0 msgr2=0x7f83ac0c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f83aa59c700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ac1562f0 0x7f83ac0c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f83aa59c700 1 -- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ac06e9d0 msgr2=0x7f83ac0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f83aa59c700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ac06e9d0 0x7f83ac0b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f83aa59c700 1 -- 172.21.15.7:0/2305097044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4009ae0 con 0x7f83ac155f20 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f83aa59c700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac155f20 0x7f83ac14f940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83a000c4e0 tx=0x7f83a000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f8392ffd700 1 -- 172.21.15.7:0/2305097044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0011590 con 0x7f83ac155f20 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f8392ffd700 1 -- 172.21.15.7:0/2305097044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a0011710 con 0x7f83ac155f20 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ac152630 con 0x7f83ac155f20 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f8392ffd700 1 -- 172.21.15.7:0/2305097044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0011ad0 con 0x7f83ac155f20 2023-10-07T07:58:02.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.673+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ac152bf0 con 0x7f83ac155f20 2023-10-07T07:58:02.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.674+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ac0716c0 con 0x7f83ac155f20 2023-10-07T07:58:02.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.675+0000 7f8392ffd700 1 -- 172.21.15.7:0/2305097044 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83a001b3b0 con 0x7f83ac155f20 2023-10-07T07:58:02.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.675+0000 7f8392ffd700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8398066f30 0x7f83980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:02.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.675+0000 7f8392ffd700 1 -- 172.21.15.7:0/2305097044 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f83a00838c0 con 0x7f83ac155f20 2023-10-07T07:58:02.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.675+0000 7f83aad9d700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8398066f30 0x7f83980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:02.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.676+0000 7f83aad9d700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8398066f30 0x7f83980693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f839c006f00 tx=0x7f839c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:02.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.680+0000 7f8392ffd700 1 -- 172.21.15.7:0/2305097044 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83a0050b30 con 0x7f83ac155f20 2023-10-07T07:58:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.884+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83ac164e50 con 0x7f83ac155f20 2023-10-07T07:58:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.885+0000 7f8392ffd700 1 -- 172.21.15.7:0/2305097044 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f83a0017070 con 0x7f83ac155f20 2023-10-07T07:58:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:02.885 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8398066f30 msgr2=0x7f83980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8398066f30 0x7f83980693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f839c006f00 tx=0x7f839c008040 comp rx=0 tx=0).stop 2023-10-07T07:58:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac155f20 msgr2=0x7f83ac14f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac155f20 0x7f83ac14f940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83a000c4e0 tx=0x7f83a000c7f0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 shutdown_connections 2023-10-07T07:58:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8398066f30 0x7f83980693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83ac1562f0 0x7f83ac0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ac06e9d0 0x7f83ac0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 --2- 172.21.15.7:0/2305097044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83ac155f20 0x7f83ac14f940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 >> 172.21.15.7:0/2305097044 conn(0x7f83ac068bd0 msgr2=0x7f83ac06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 shutdown_connections 2023-10-07T07:58:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:02.887+0000 7f83b11e7700 1 -- 172.21.15.7:0/2305097044 wait complete. 2023-10-07T07:58:02.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T07:57:48.285077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T07:57:49.111979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T07:58:02.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:58:03.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.037+0000 7f23c8211700 1 Processor -- start 2023-10-07T07:58:03.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.037+0000 7f23c8211700 1 -- start start 2023-10-07T07:58:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c8211700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c0155d90 0x7f23c0156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c8211700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23c00c7a50 0x7f23c006c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c8211700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c006b6f0 con 0x7f23c00c75e0 2023-10-07T07:58:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c8211700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c00cbbf0 con 0x7f23c00c7a50 2023-10-07T07:58:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c8211700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c006bf70 con 0x7f23c0155d90 2023-10-07T07:58:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c67ae700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23c00c7a50 0x7f23c006c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c67ae700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23c00c7a50 0x7f23c006c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48030/0 (socket says 172.21.15.7:48030) 2023-10-07T07:58:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c67ae700 1 -- 172.21.15.7:0/2190488136 learned_addr learned my addr 172.21.15.7:0/2190488136 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c5fad700 1 --2- 172.21.15.7:0/2190488136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c0155d90 0x7f23c0156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c67ae700 1 -- 172.21.15.7:0/2190488136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c0155d90 msgr2=0x7f23c0156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c67ae700 1 --2- 172.21.15.7:0/2190488136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c0155d90 0x7f23c0156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c67ae700 1 -- 172.21.15.7:0/2190488136 >> v1:172.21.15.172:6789/0 conn(0x7f23c00c75e0 legacy=0x7f23c006e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.038+0000 7f23c67ae700 1 -- 172.21.15.7:0/2190488136 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c00cb910 con 0x7f23c00c7a50 2023-10-07T07:58:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c67ae700 1 --2- 172.21.15.7:0/2190488136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c00c7a50 0x7f23c006c400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009f20 tx=0x7f23b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2c742a4e4c056c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c4fab700 1 -- 172.21.15.7:0/2190488136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b800e070 con 0x7f23c00c7a50 2023-10-07T07:58:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c4fab700 1 -- 172.21.15.7:0/2190488136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b800b9d0 con 0x7f23c00c7a50 2023-10-07T07:58:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c4fab700 1 -- 172.21.15.7:0/2190488136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b80092e0 con 0x7f23c00c7a50 2023-10-07T07:58:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c8211700 1 -- 172.21.15.7:0/2190488136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c00c7a50 msgr2=0x7f23c006c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c8211700 1 --2- 172.21.15.7:0/2190488136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c00c7a50 0x7f23c006c400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009f20 tx=0x7f23b8009840 comp rx=0 tx=0).stop 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c8211700 1 -- 172.21.15.7:0/2190488136 shutdown_connections 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c8211700 1 --2- 172.21.15.7:0/2190488136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c00c7a50 0x7f23c006c400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c8211700 1 --2- 172.21.15.7:0/2190488136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c0155d90 0x7f23c0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c8211700 1 -- 172.21.15.7:0/2190488136 >> 172.21.15.7:0/2190488136 conn(0x7f23c0068100 msgr2=0x7f23c0068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.039+0000 7f23c8211700 1 -- 172.21.15.7:0/2190488136 shutdown_connections 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.040+0000 7f23c8211700 1 -- 172.21.15.7:0/2190488136 wait complete. 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.041+0000 7f23c8211700 1 Processor -- start 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.041+0000 7f23c8211700 1 -- start start 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.041+0000 7f23c8211700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23c00c75e0 0x7f23c0150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.041+0000 7f23c8211700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c00c7a50 0x7f23c00ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.041+0000 7f23c8211700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0155d90 0x7f23c00b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.041+0000 7f23c8211700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c00af140 con 0x7f23c0155d90 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.041+0000 7f23c8211700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0157460 con 0x7f23c00c75e0 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.041+0000 7f23c8211700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c00c4050 con 0x7f23c00c7a50 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c67ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0155d90 0x7f23c00b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c67ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0155d90 0x7f23c00b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48032/0 (socket says 172.21.15.7:48032) 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c67ae700 1 -- 172.21.15.7:0/3076480636 learned_addr learned my addr 172.21.15.7:0/3076480636 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c5fad700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23c00c75e0 0x7f23c0150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c57ac700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c00c7a50 0x7f23c00ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c67ae700 1 -- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c00c7a50 msgr2=0x7f23c00ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c67ae700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c00c7a50 0x7f23c00ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c67ae700 1 -- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23c00c75e0 msgr2=0x7f23c0150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c67ae700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23c00c75e0 0x7f23c0150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c67ae700 1 -- 172.21.15.7:0/3076480636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b8009ae0 con 0x7f23c0155d90 2023-10-07T07:58:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c67ae700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0155d90 0x7f23c00b00f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f23b800b610 tx=0x7f23b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23b5ffb700 1 -- 172.21.15.7:0/3076480636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8008b40 con 0x7f23c0155d90 2023-10-07T07:58:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23b5ffb700 1 -- 172.21.15.7:0/3076480636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b8008ca0 con 0x7f23c0155d90 2023-10-07T07:58:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c00aeb80 con 0x7f23c0155d90 2023-10-07T07:58:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.042+0000 7f23b5ffb700 1 -- 172.21.15.7:0/3076480636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8026660 con 0x7f23c0155d90 2023-10-07T07:58:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.043+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c01570f0 con 0x7f23c0155d90 2023-10-07T07:58:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.043+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c01575f0 con 0x7f23c0155d90 2023-10-07T07:58:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.044+0000 7f23b5ffb700 1 -- 172.21.15.7:0/3076480636 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f23b8026a90 con 0x7f23c0155d90 2023-10-07T07:58:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.044+0000 7f23b5ffb700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f23a4066f30 0x7f23a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.045+0000 7f23b5ffb700 1 -- 172.21.15.7:0/3076480636 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f23b8085f50 con 0x7f23c0155d90 2023-10-07T07:58:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.045+0000 7f23c5fad700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f23a4066f30 0x7f23a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.045+0000 7f23c5fad700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f23a4066f30 0x7f23a40693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009d10 tx=0x7f23bc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:03.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.049+0000 7f23b5ffb700 1 -- 172.21.15.7:0/3076480636 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23b80531c0 con 0x7f23c0155d90 2023-10-07T07:58:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.249+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23c004c3c0 con 0x7f23c0155d90 2023-10-07T07:58:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.249+0000 7f23b5ffb700 1 -- 172.21.15.7:0/3076480636 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23b80567e0 con 0x7f23c0155d90 2023-10-07T07:58:03.250 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f23a4066f30 msgr2=0x7f23a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f23a4066f30 0x7f23a40693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009d10 tx=0x7f23bc0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0155d90 msgr2=0x7f23c00b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0155d90 0x7f23c00b00f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f23b800b610 tx=0x7f23b8018000 comp rx=0 tx=0).stop 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 shutdown_connections 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f23a4066f30 0x7f23a40693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0155d90 0x7f23c00b00f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23c00c75e0 0x7f23c0150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 --2- 172.21.15.7:0/3076480636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23c00c7a50 0x7f23c00ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.251+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 >> 172.21.15.7:0/3076480636 conn(0x7f23c0068100 msgr2=0x7f23c006c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.252+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 shutdown_connections 2023-10-07T07:58:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.252+0000 7f23c8211700 1 -- 172.21.15.7:0/3076480636 wait complete. 2023-10-07T07:58:03.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T07:58:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.400+0000 7f1efbdd1700 1 Processor -- start 2023-10-07T07:58:03.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.401+0000 7f1efbdd1700 1 -- start start 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.401+0000 7f1efbdd1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ef4067240 0x7f1ef415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.401+0000 7f1efbdd1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ef4067610 0x7f1ef4153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.401+0000 7f1efbdd1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef406f320 con 0x7f1ef41539c0 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.401+0000 7f1efbdd1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef406a500 con 0x7f1ef4067240 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1efbdd1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef4153db0 con 0x7f1ef4067610 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef9b6d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ef4067240 0x7f1ef415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1efa36e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ef41539c0 0x7f1ef40aa980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53080/0 (socket says 172.21.15.7:53080) 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1efa36e700 1 -- 172.21.15.7:0/3816540663 learned_addr learned my addr 172.21.15.7:0/3816540663 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef9b6d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ef4067240 0x7f1ef415a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48046/0 (socket says 172.21.15.7:48046) 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef936c700 1 --2- 172.21.15.7:0/3816540663 >> v2:172.21.15.172:3300/0 conn(0x7f1ef4067610 0x7f1ef4153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef9b6d700 1 -- 172.21.15.7:0/3816540663 >> v2:172.21.15.172:3300/0 conn(0x7f1ef4067610 msgr2=0x7f1ef4153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef9b6d700 1 --2- 172.21.15.7:0/3816540663 >> v2:172.21.15.172:3300/0 conn(0x7f1ef4067610 0x7f1ef4153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef9b6d700 1 -- 172.21.15.7:0/3816540663 >> v1:172.21.15.7:6789/0 conn(0x7f1ef41539c0 legacy=0x7f1ef40aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef9b6d700 1 -- 172.21.15.7:0/3816540663 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef406e7a0 con 0x7f1ef4067240 2023-10-07T07:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef8b6b700 1 -- 172.21.15.7:0/3816540663 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2387159798 0 0) 0x7f1ef406f320 con 0x7f1ef41539c0 2023-10-07T07:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef9b6d700 1 --2- 172.21.15.7:0/3816540663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef4067240 0x7f1ef415a2d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ef406f930 tx=0x7f1ee40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52894e521ef0035 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef8b6b700 1 -- 172.21.15.7:0/3816540663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4011630 con 0x7f1ef4067240 2023-10-07T07:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef8b6b700 1 -- 172.21.15.7:0/3816540663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee4011790 con 0x7f1ef4067240 2023-10-07T07:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.402+0000 7f1ef8b6b700 1 -- 172.21.15.7:0/3816540663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4011b30 con 0x7f1ef4067240 2023-10-07T07:58:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.403+0000 7f1efbdd1700 1 -- 172.21.15.7:0/3816540663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef4067240 msgr2=0x7f1ef415a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.403+0000 7f1efbdd1700 1 --2- 172.21.15.7:0/3816540663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef4067240 0x7f1ef415a2d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ef406f930 tx=0x7f1ee40091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.403+0000 7f1efbdd1700 1 -- 172.21.15.7:0/3816540663 shutdown_connections 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.403+0000 7f1efbdd1700 1 --2- 172.21.15.7:0/3816540663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef4067240 0x7f1ef415a2d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.403+0000 7f1efbdd1700 1 --2- 172.21.15.7:0/3816540663 >> v2:172.21.15.172:3300/0 conn(0x7f1ef4067610 0x7f1ef4153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.403+0000 7f1efbdd1700 1 -- 172.21.15.7:0/3816540663 >> 172.21.15.7:0/3816540663 conn(0x7f1ef4068100 msgr2=0x7f1ef4068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.403+0000 7f1efbdd1700 1 -- 172.21.15.7:0/3816540663 shutdown_connections 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.403+0000 7f1efbdd1700 1 -- 172.21.15.7:0/3816540663 wait complete. 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.404+0000 7f1efbdd1700 1 Processor -- start 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.404+0000 7f1efbdd1700 1 -- start start 2023-10-07T07:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1efbdd1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ef4067240 0x7f1ef40b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1efbdd1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ef4067610 0x7f1ef4169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1efbdd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef41539c0 0x7f1ef40b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1efbdd1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef40b10c0 con 0x7f1ef41539c0 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1efbdd1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef40b1930 con 0x7f1ef4067610 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1efbdd1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef40b1c10 con 0x7f1ef4067240 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1efa36e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef41539c0 0x7f1ef40b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1efa36e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef41539c0 0x7f1ef40b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48058/0 (socket says 172.21.15.7:48058) 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1efa36e700 1 -- 172.21.15.7:0/2152811336 learned_addr learned my addr 172.21.15.7:0/2152811336 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1ef936c700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ef4067610 0x7f1ef4169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.405+0000 7f1ef9b6d700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ef4067240 0x7f1ef40b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1efa36e700 1 -- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ef4067240 msgr2=0x7f1ef40b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1efa36e700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ef4067240 0x7f1ef40b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1efa36e700 1 -- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ef4067610 msgr2=0x7f1ef4169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1efa36e700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ef4067610 0x7f1ef4169b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1efa36e700 1 -- 172.21.15.7:0/2152811336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee4009420 con 0x7f1ef41539c0 2023-10-07T07:58:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1efa36e700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef41539c0 0x7f1ef40b0700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00cbf0 tx=0x7f1eec0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1ee9ffb700 1 -- 172.21.15.7:0/2152811336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00a910 con 0x7f1ef41539c0 2023-10-07T07:58:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1ee9ffb700 1 -- 172.21.15.7:0/2152811336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec00aa70 con 0x7f1ef41539c0 2023-10-07T07:58:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef40b1ef0 con 0x7f1ef41539c0 2023-10-07T07:58:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1ee9ffb700 1 -- 172.21.15.7:0/2152811336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00ae30 con 0x7f1ef41539c0 2023-10-07T07:58:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.406+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef40b2c30 con 0x7f1ef41539c0 2023-10-07T07:58:03.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.407+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef4159b40 con 0x7f1ef41539c0 2023-10-07T07:58:03.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.408+0000 7f1ee9ffb700 1 -- 172.21.15.7:0/2152811336 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1eec007300 con 0x7f1ef41539c0 2023-10-07T07:58:03.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.408+0000 7f1ee9ffb700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1ed8066f60 0x7f1ed8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.408+0000 7f1ee9ffb700 1 -- 172.21.15.7:0/2152811336 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1eec084870 con 0x7f1ef41539c0 2023-10-07T07:58:03.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.408+0000 7f1ef9b6d700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1ed8066f60 0x7f1ed8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.409+0000 7f1ef9b6d700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1ed8066f60 0x7f1ed8069410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40093f0 tx=0x7f1ee4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:03.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.412+0000 7f1ee9ffb700 1 -- 172.21.15.7:0/2152811336 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1eec051b60 con 0x7f1ef41539c0 2023-10-07T07:58:03.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.609+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ef404c3c0 con 0x7f1ef41539c0 2023-10-07T07:58:03.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.610+0000 7f1ee9ffb700 1 -- 172.21.15.7:0/2152811336 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1eec055180 con 0x7f1ef41539c0 2023-10-07T07:58:03.611 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-07T07:58:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1ed8066f60 msgr2=0x7f1ed8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1ed8066f60 0x7f1ed8069410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40093f0 tx=0x7f1ee4017000 comp rx=0 tx=0).stop 2023-10-07T07:58:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef41539c0 msgr2=0x7f1ef40b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef41539c0 0x7f1ef40b0700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00cbf0 tx=0x7f1eec0019b0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 shutdown_connections 2023-10-07T07:58:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1ed8066f60 0x7f1ed8069410 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ef41539c0 0x7f1ef40b0700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ef4067240 0x7f1ef40b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 --2- 172.21.15.7:0/2152811336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ef4067610 0x7f1ef4169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 >> 172.21.15.7:0/2152811336 conn(0x7f1ef4068100 msgr2=0x7f1ef40c2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 shutdown_connections 2023-10-07T07:58:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.612+0000 7f1efbdd1700 1 -- 172.21.15.7:0/2152811336 wait complete. 2023-10-07T07:58:03.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T07:58:03.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.760+0000 7f4089e3d700 1 Processor -- start 2023-10-07T07:58:03.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.760+0000 7f4089e3d700 1 -- start start 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4089e3d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40840cc2b0 0x7f4084068000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4089e3d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 0x7f40840ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4089e3d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840689a0 con 0x7f40840cbee0 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4089e3d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40840c8360 con 0x7f40840cc2b0 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4089e3d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40840c7480 con 0x7f4084068540 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4082ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40840cc2b0 0x7f4084068000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4082ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40840cc2b0 0x7f4084068000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48062/0 (socket says 172.21.15.7:48062) 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4082ffd700 1 -- 172.21.15.7:0/130875055 learned_addr learned my addr 172.21.15.7:0/130875055 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4083fff700 1 --2- 172.21.15.7:0/130875055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 0x7f40840ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4082ffd700 1 -- 172.21.15.7:0/130875055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 msgr2=0x7f40840ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4082ffd700 1 --2- 172.21.15.7:0/130875055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 0x7f40840ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4082ffd700 1 -- 172.21.15.7:0/130875055 >> v1:172.21.15.172:6789/0 conn(0x7f40840cbee0 legacy=0x7f40840c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4082ffd700 1 -- 172.21.15.7:0/130875055 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840c7750 con 0x7f40840cc2b0 2023-10-07T07:58:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4083fff700 1 --2- 172.21.15.7:0/130875055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 0x7f40840ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f4082ffd700 1 --2- 172.21.15.7:0/130875055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 0x7f4084068000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f40780099b0 tx=0x7f4078009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bb949a4d4721ee server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.761+0000 7f40827fc700 1 -- 172.21.15.7:0/130875055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407800b860 con 0x7f40840cc2b0 2023-10-07T07:58:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f40827fc700 1 -- 172.21.15.7:0/130875055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407800b9c0 con 0x7f40840cc2b0 2023-10-07T07:58:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f40827fc700 1 -- 172.21.15.7:0/130875055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407800bd60 con 0x7f40840cc2b0 2023-10-07T07:58:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f4089e3d700 1 -- 172.21.15.7:0/130875055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 msgr2=0x7f4084068000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f4089e3d700 1 --2- 172.21.15.7:0/130875055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 0x7f4084068000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f40780099b0 tx=0x7f4078009250 comp rx=0 tx=0).stop 2023-10-07T07:58:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f4089e3d700 1 -- 172.21.15.7:0/130875055 shutdown_connections 2023-10-07T07:58:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f4089e3d700 1 --2- 172.21.15.7:0/130875055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 0x7f40840ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f4089e3d700 1 --2- 172.21.15.7:0/130875055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 0x7f4084068000 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f4089e3d700 1 -- 172.21.15.7:0/130875055 >> 172.21.15.7:0/130875055 conn(0x7f40840c9c30 msgr2=0x7f408406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f4089e3d700 1 -- 172.21.15.7:0/130875055 shutdown_connections 2023-10-07T07:58:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.762+0000 7f4089e3d700 1 -- 172.21.15.7:0/130875055 wait complete. 2023-10-07T07:58:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.764+0000 7f4089e3d700 1 Processor -- start 2023-10-07T07:58:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.764+0000 7f4089e3d700 1 -- start start 2023-10-07T07:58:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4089e3d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 0x7f4084151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4089e3d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40840cbee0 0x7f4084137150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4089e3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 0x7f408414d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4089e3d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408414c280 con 0x7f40840cc2b0 2023-10-07T07:58:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4089e3d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40841549d0 con 0x7f40840cbee0 2023-10-07T07:58:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4089e3d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4084153dc0 con 0x7f4084068540 2023-10-07T07:58:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4083fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 0x7f408414d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4083fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 0x7f408414d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48076/0 (socket says 172.21.15.7:48076) 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4083fff700 1 -- 172.21.15.7:0/1510992199 learned_addr learned my addr 172.21.15.7:0/1510992199 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4082ffd700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40840cbee0 0x7f4084137150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f40837fe700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 0x7f4084151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4083fff700 1 -- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 msgr2=0x7f4084151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4083fff700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 0x7f4084151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4083fff700 1 -- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40840cbee0 msgr2=0x7f4084137150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4083fff700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40840cbee0 0x7f4084137150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4083fff700 1 -- 172.21.15.7:0/1510992199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780094f0 con 0x7f40840cc2b0 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f4083fff700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 0x7f408414d2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f407c00d180 tx=0x7f407c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.765+0000 7f406b7fe700 1 -- 172.21.15.7:0/1510992199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c00dcc0 con 0x7f40840cc2b0 2023-10-07T07:58:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.766+0000 7f406b7fe700 1 -- 172.21.15.7:0/1510992199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c00f040 con 0x7f40840cc2b0 2023-10-07T07:58:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.766+0000 7f406b7fe700 1 -- 172.21.15.7:0/1510992199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c015690 con 0x7f40840cc2b0 2023-10-07T07:58:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.766+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40841540a0 con 0x7f40840cc2b0 2023-10-07T07:58:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.766+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4084154660 con 0x7f40840cc2b0 2023-10-07T07:58:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.767+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408404c3c0 con 0x7f40840cc2b0 2023-10-07T07:58:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.767+0000 7f406b7fe700 1 -- 172.21.15.7:0/1510992199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f407c01e3b0 con 0x7f40840cc2b0 2023-10-07T07:58:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.768+0000 7f406b7fe700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4070066f30 0x7f40700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.768+0000 7f406b7fe700 1 -- 172.21.15.7:0/1510992199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f407c084a40 con 0x7f40840cc2b0 2023-10-07T07:58:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.768+0000 7f40837fe700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4070066f30 0x7f40700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.768+0000 7f40837fe700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4070066f30 0x7f40700693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4074007fd0 tx=0x7f4074007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.772+0000 7f406b7fe700 1 -- 172.21.15.7:0/1510992199 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f407c086e60 con 0x7f40840cc2b0 2023-10-07T07:58:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.970+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4084154b10 con 0x7f40840cc2b0 2023-10-07T07:58:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.970+0000 7f406b7fe700 1 -- 172.21.15.7:0/1510992199 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f407c01a0a0 con 0x7f40840cc2b0 2023-10-07T07:58:03.972 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-07T07:58:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4070066f30 msgr2=0x7f40700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4070066f30 0x7f40700693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4074007fd0 tx=0x7f4074007eb0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 msgr2=0x7f408414d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 0x7f408414d2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f407c00d180 tx=0x7f407c009a50 comp rx=0 tx=0).stop 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 shutdown_connections 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4070066f30 0x7f40700693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40840cc2b0 0x7f408414d2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4084068540 0x7f4084151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 --2- 172.21.15.7:0/1510992199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40840cbee0 0x7f4084137150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 >> 172.21.15.7:0/1510992199 conn(0x7f40840c9c30 msgr2=0x7f4084150220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 shutdown_connections 2023-10-07T07:58:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:03.972+0000 7f4089e3d700 1 -- 172.21.15.7:0/1510992199 wait complete. 2023-10-07T07:58:03.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:58:04.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.118+0000 7f375e969700 1 Processor -- start 2023-10-07T07:58:04.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.118+0000 7f375e969700 1 -- start start 2023-10-07T07:58:04.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f375e969700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 0x7f375806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f375e969700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375815a330 con 0x7f37580c9af0 2023-10-07T07:58:04.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f375e969700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375815a610 con 0x7f37580c9ec0 2023-10-07T07:58:04.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f375e969700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37580b03a0 con 0x7f3758063230 2023-10-07T07:58:04.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f3757fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f37580c9af0 0x7f375815d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53096/0 (socket says 172.21.15.7:53096) 2023-10-07T07:58:04.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f3757fff700 1 -- 172.21.15.7:0/3293100138 learned_addr learned my addr 172.21.15.7:0/3293100138 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f37577fe700 1 --2- 172.21.15.7:0/3293100138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 0x7f375806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f3756ffd700 1 -- 172.21.15.7:0/3293100138 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174075793 0 0) 0x7f375815a330 con 0x7f37580c9af0 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f3756ffd700 1 -- 172.21.15.7:0/3293100138 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3740003570 con 0x7f37580c9af0 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f3756ffd700 1 -- 172.21.15.7:0/3293100138 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144595619 0 0) 0x7f375815a610 con 0x7f37580c9ec0 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f3756ffd700 1 -- 172.21.15.7:0/3293100138 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f375815a330 con 0x7f37580c9ec0 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f37577fe700 1 -- 172.21.15.7:0/3293100138 >> v1:172.21.15.172:6789/0 conn(0x7f37580c9ec0 legacy=0x7f375806e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.119+0000 7f3756ffd700 1 -- 172.21.15.7:0/3293100138 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2120703179 0 0) 0x7f3740003570 con 0x7f37580c9af0 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.120+0000 7f37577fe700 1 -- 172.21.15.7:0/3293100138 >> v1:172.21.15.7:6789/0 conn(0x7f37580c9af0 legacy=0x7f375815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.120+0000 7f37577fe700 1 -- 172.21.15.7:0/3293100138 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375815a8f0 con 0x7f3758063230 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.120+0000 7f37577fe700 1 --2- 172.21.15.7:0/3293100138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 0x7f375806db10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f374c009910 tx=0x7f374c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed459a7f345409d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.120+0000 7f3756ffd700 1 -- 172.21.15.7:0/3293100138 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748001470 con 0x7f3758063230 2023-10-07T07:58:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.120+0000 7f3756ffd700 1 -- 172.21.15.7:0/3293100138 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c011420 con 0x7f3758063230 2023-10-07T07:58:04.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.120+0000 7f3756ffd700 1 -- 172.21.15.7:0/3293100138 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c0117c0 con 0x7f3758063230 2023-10-07T07:58:04.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.120+0000 7f375e969700 1 -- 172.21.15.7:0/3293100138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 msgr2=0x7f375806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.120+0000 7f375e969700 1 --2- 172.21.15.7:0/3293100138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 0x7f375806db10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f374c009910 tx=0x7f374c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.121+0000 7f375e969700 1 -- 172.21.15.7:0/3293100138 shutdown_connections 2023-10-07T07:58:04.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.121+0000 7f375e969700 1 --2- 172.21.15.7:0/3293100138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 0x7f375806db10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.121+0000 7f375e969700 1 -- 172.21.15.7:0/3293100138 >> 172.21.15.7:0/3293100138 conn(0x7f37580cd1a0 msgr2=0x7f37580c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:04.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.121+0000 7f375e969700 1 -- 172.21.15.7:0/3293100138 shutdown_connections 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.121+0000 7f375e969700 1 -- 172.21.15.7:0/3293100138 wait complete. 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.122+0000 7f375e969700 1 Processor -- start 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.122+0000 7f375e969700 1 -- start start 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375e969700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 0x7f37581451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375e969700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37580c9af0 0x7f37580c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375e969700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37580c9ec0 0x7f37580c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375e969700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37580c5ce0 con 0x7f37580c9ec0 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375e969700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3758145cd0 con 0x7f37580c9af0 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375e969700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37580c65b0 con 0x7f3758063230 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375cf06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37580c9ec0 0x7f37580c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375cf06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37580c9ec0 0x7f37580c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48086/0 (socket says 172.21.15.7:48086) 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375cf06700 1 -- 172.21.15.7:0/3135151216 learned_addr learned my addr 172.21.15.7:0/3135151216 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f3757fff700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 0x7f37581451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f37577fe700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37580c9af0 0x7f37580c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375cf06700 1 -- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 msgr2=0x7f37581451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375cf06700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 0x7f37581451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375cf06700 1 -- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37580c9af0 msgr2=0x7f37580c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375cf06700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37580c9af0 0x7f37580c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f375cf06700 1 -- 172.21.15.7:0/3135151216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c009420 con 0x7f37580c9ec0 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.123+0000 7f37577fe700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37580c9af0 0x7f37580c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.124+0000 7f375cf06700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37580c9ec0 0x7f37580c83d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f375000c910 tx=0x7f37500027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.124+0000 7f373ffff700 1 -- 172.21.15.7:0/3135151216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750012630 con 0x7f37580c9ec0 2023-10-07T07:58:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.124+0000 7f373ffff700 1 -- 172.21.15.7:0/3135151216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37500127b0 con 0x7f37580c9ec0 2023-10-07T07:58:04.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.124+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3758145710 con 0x7f37580c9ec0 2023-10-07T07:58:04.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.124+0000 7f373ffff700 1 -- 172.21.15.7:0/3135151216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750012b70 con 0x7f37580c9ec0 2023-10-07T07:58:04.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.124+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375815c4f0 con 0x7f37580c9ec0 2023-10-07T07:58:04.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.125+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37580716c0 con 0x7f37580c9ec0 2023-10-07T07:58:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.125+0000 7f373ffff700 1 -- 172.21.15.7:0/3135151216 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f375001b400 con 0x7f37580c9ec0 2023-10-07T07:58:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.126+0000 7f373ffff700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3744066f30 0x7f37440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.126+0000 7f373ffff700 1 -- 172.21.15.7:0/3135151216 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f37500848f0 con 0x7f37580c9ec0 2023-10-07T07:58:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.126+0000 7f3757fff700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3744066f30 0x7f37440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.126+0000 7f3757fff700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3744066f30 0x7f37440693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f37480011e0 tx=0x7f3748000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:04.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.130+0000 7f373ffff700 1 -- 172.21.15.7:0/3135151216 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3750051c10 con 0x7f37580c9ec0 2023-10-07T07:58:04.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.378+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37580c66f0 con 0x7f37580c9ec0 2023-10-07T07:58:04.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.379+0000 7f373ffff700 1 -- 172.21.15.7:0/3135151216 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f3750055230 con 0x7f37580c9ec0 2023-10-07T07:58:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:04.380 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2318752844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2318752844},{"type":"v1","addr":"172.21.15.172:6836","nonce":2318752844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1231430544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1231430544},{"type":"v1","addr":"172.21.15.172:6837","nonce":1231430544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1421504798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1421504798},{"type":"v1","addr":"172.21.15.7:6833","nonce":1421504798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:57:50.139330+0000","modified":"2023-10-07T07:57:53.727131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T07:58:04.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3744066f30 msgr2=0x7f37440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3744066f30 0x7f37440693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f37480011e0 tx=0x7f3748000bc0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37580c9ec0 msgr2=0x7f37580c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37580c9ec0 0x7f37580c83d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f375000c910 tx=0x7f37500027a0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 shutdown_connections 2023-10-07T07:58:04.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3744066f30 0x7f37440693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37580c9ec0 0x7f37580c83d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3758063230 0x7f37581451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 --2- 172.21.15.7:0/3135151216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37580c9af0 0x7f37580c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 >> 172.21.15.7:0/3135151216 conn(0x7f37580cd1a0 msgr2=0x7f3758159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:04.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 shutdown_connections 2023-10-07T07:58:04.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.381+0000 7f375e969700 1 -- 172.21.15.7:0/3135151216 wait complete. 2023-10-07T07:58:04.384 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2023-10-07T07:58:04.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:04.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.527+0000 7f6e09fd1700 1 Processor -- start 2023-10-07T07:58:04.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.527+0000 7f6e09fd1700 1 -- start start 2023-10-07T07:58:04.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e09fd1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 0x7f6e0406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e09fd1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0415a330 con 0x7f6e040c9af0 2023-10-07T07:58:04.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e09fd1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0415a610 con 0x7f6e040c9ec0 2023-10-07T07:58:04.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e09fd1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040b03a0 con 0x7f6e04063230 2023-10-07T07:58:04.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e037fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6e040c9af0 0x7f6e0415d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53098/0 (socket says 172.21.15.7:53098) 2023-10-07T07:58:04.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e037fe700 1 -- 172.21.15.7:0/3002412606 learned_addr learned my addr 172.21.15.7:0/3002412606 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:04.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e02ffd700 1 --2- 172.21.15.7:0/3002412606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 0x7f6e0406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e027fc700 1 -- 172.21.15.7:0/3002412606 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429364221 0 0) 0x7f6e0415a330 con 0x7f6e040c9af0 2023-10-07T07:58:04.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e027fc700 1 -- 172.21.15.7:0/3002412606 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dec003570 con 0x7f6e040c9af0 2023-10-07T07:58:04.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e027fc700 1 -- 172.21.15.7:0/3002412606 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700404912 0 0) 0x7f6e0415a610 con 0x7f6e040c9ec0 2023-10-07T07:58:04.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.528+0000 7f6e027fc700 1 -- 172.21.15.7:0/3002412606 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e0415a330 con 0x7f6e040c9ec0 2023-10-07T07:58:04.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e02ffd700 1 -- 172.21.15.7:0/3002412606 >> v1:172.21.15.172:6789/0 conn(0x7f6e040c9ec0 legacy=0x7f6e0406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e027fc700 1 -- 172.21.15.7:0/3002412606 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2602479459 0 0) 0x7f6dec003570 con 0x7f6e040c9af0 2023-10-07T07:58:04.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e02ffd700 1 -- 172.21.15.7:0/3002412606 >> v1:172.21.15.7:6789/0 conn(0x7f6e040c9af0 legacy=0x7f6e0415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e02ffd700 1 -- 172.21.15.7:0/3002412606 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0415a8f0 con 0x7f6e04063230 2023-10-07T07:58:04.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e02ffd700 1 --2- 172.21.15.7:0/3002412606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 0x7f6e0406db10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009910 tx=0x7f6df80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4d94ba1588de8f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:04.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e027fc700 1 -- 172.21.15.7:0/3002412606 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4001470 con 0x7f6e04063230 2023-10-07T07:58:04.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e027fc700 1 -- 172.21.15.7:0/3002412606 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df8011420 con 0x7f6e04063230 2023-10-07T07:58:04.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e027fc700 1 -- 172.21.15.7:0/3002412606 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df80117c0 con 0x7f6e04063230 2023-10-07T07:58:04.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3002412606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 msgr2=0x7f6e0406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.529+0000 7f6e09fd1700 1 --2- 172.21.15.7:0/3002412606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 0x7f6e0406db10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009910 tx=0x7f6df80091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.530+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3002412606 shutdown_connections 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.530+0000 7f6e09fd1700 1 --2- 172.21.15.7:0/3002412606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 0x7f6e0406db10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.530+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3002412606 >> 172.21.15.7:0/3002412606 conn(0x7f6e040cd1a0 msgr2=0x7f6e040c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.530+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3002412606 shutdown_connections 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.530+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3002412606 wait complete. 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.531+0000 7f6e09fd1700 1 Processor -- start 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.531+0000 7f6e09fd1700 1 -- start start 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e09fd1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 0x7f6e040a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e09fd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e040c9af0 0x7f6e040c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e09fd1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c9ec0 0x7f6e040a7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e09fd1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040c7d50 con 0x7f6e040c9af0 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e09fd1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04150cc0 con 0x7f6e040c9ec0 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e09fd1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040c7140 con 0x7f6e04063230 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e040c9af0 0x7f6e040c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e040c9af0 0x7f6e040c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48094/0 (socket says 172.21.15.7:48094) 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e02ffd700 1 -- 172.21.15.7:0/3117707067 learned_addr learned my addr 172.21.15.7:0/3117707067 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e037fe700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 0x7f6e040a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e03fff700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c9ec0 0x7f6e040a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e02ffd700 1 -- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 msgr2=0x7f6e040a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e02ffd700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 0x7f6e040a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e02ffd700 1 -- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c9ec0 msgr2=0x7f6e040a7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e02ffd700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c9ec0 0x7f6e040a7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e02ffd700 1 -- 172.21.15.7:0/3117707067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8009420 con 0x7f6e040c9af0 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.532+0000 7f6e02ffd700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e040c9af0 0x7f6e040c68f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6df80091e0 tx=0x7f6df8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.533+0000 7f6deb7fe700 1 -- 172.21.15.7:0/3117707067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df801cb30 con 0x7f6e040c9af0 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.533+0000 7f6deb7fe700 1 -- 172.21.15.7:0/3117707067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df801cc90 con 0x7f6e040c9af0 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.533+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e040c7420 con 0x7f6e040c9af0 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.533+0000 7f6deb7fe700 1 -- 172.21.15.7:0/3117707067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df801a6a0 con 0x7f6e040c9af0 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.533+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e040c79e0 con 0x7f6e040c9af0 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.534+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e040c7ec0 con 0x7f6e040c9af0 2023-10-07T07:58:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.534+0000 7f6deb7fe700 1 -- 172.21.15.7:0/3117707067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6df801aad0 con 0x7f6e040c9af0 2023-10-07T07:58:04.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.534+0000 7f6deb7fe700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6df0066f30 0x7f6df00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.535+0000 7f6deb7fe700 1 -- 172.21.15.7:0/3117707067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6df8086280 con 0x7f6e040c9af0 2023-10-07T07:58:04.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.535+0000 7f6e037fe700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6df0066f30 0x7f6df00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.535+0000 7f6e037fe700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6df0066f30 0x7f6df00693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df40011e0 tx=0x7f6df4001620 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:04.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.539+0000 7f6deb7fe700 1 -- 172.21.15.7:0/3117707067 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6df80534f0 con 0x7f6e040c9af0 2023-10-07T07:58:04.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.744+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e040716c0 con 0x7f6e040c9af0 2023-10-07T07:58:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.745+0000 7f6deb7fe700 1 -- 172.21.15.7:0/3117707067 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f6df8056b10 con 0x7f6e040c9af0 2023-10-07T07:58:04.746 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:04.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6df0066f30 msgr2=0x7f6df00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6df0066f30 0x7f6df00693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df40011e0 tx=0x7f6df4001620 comp rx=0 tx=0).stop 2023-10-07T07:58:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e040c9af0 msgr2=0x7f6e040c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e040c9af0 0x7f6e040c68f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6df80091e0 tx=0x7f6df8016000 comp rx=0 tx=0).stop 2023-10-07T07:58:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 shutdown_connections 2023-10-07T07:58:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6df0066f30 0x7f6df00693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e040c9ec0 0x7f6e040a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e04063230 0x7f6e040a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 --2- 172.21.15.7:0/3117707067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e040c9af0 0x7f6e040c68f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 >> 172.21.15.7:0/3117707067 conn(0x7f6e040cd1a0 msgr2=0x7f6e04158120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:04.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 shutdown_connections 2023-10-07T07:58:04.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.748+0000 7f6e09fd1700 1 -- 172.21.15.7:0/3117707067 wait complete. 2023-10-07T07:58:04.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:58:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.894+0000 7fbd093d7700 1 Processor -- start 2023-10-07T07:58:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.894+0000 7fbd093d7700 1 -- start start 2023-10-07T07:58:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.894+0000 7fbd093d7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd04159380 0x7fbd040b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.894+0000 7fbd093d7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd040b4820 con 0x7fbd040d0250 2023-10-07T07:58:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.894+0000 7fbd093d7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd040b5000 con 0x7fbd040b4270 2023-10-07T07:58:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.894+0000 7fbd093d7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd040b52e0 con 0x7fbd04159380 2023-10-07T07:58:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd02ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd04159380 0x7fbd040b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd02ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd04159380 0x7fbd040b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48110/0 (socket says 172.21.15.7:48110) 2023-10-07T07:58:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd027fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbd040d0250 0x7fbd040b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53102/0 (socket says 172.21.15.7:53102) 2023-10-07T07:58:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd02ffd700 1 -- 172.21.15.7:0/3163133894 learned_addr learned my addr 172.21.15.7:0/3163133894 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd02ffd700 1 -- 172.21.15.7:0/3163133894 >> v1:172.21.15.172:6789/0 conn(0x7fbd040b4270 legacy=0x7fbd040ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd02ffd700 1 -- 172.21.15.7:0/3163133894 >> v1:172.21.15.7:6789/0 conn(0x7fbd040d0250 legacy=0x7fbd040b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd02ffd700 1 -- 172.21.15.7:0/3163133894 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd040b55c0 con 0x7fbd04159380 2023-10-07T07:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd01ffb700 1 -- 172.21.15.7:0/3163133894 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486969303 0 0) 0x7fbd040b4820 con 0x7fbd040d0250 2023-10-07T07:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd02ffd700 1 --2- 172.21.15.7:0/3163133894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd04159380 0x7fbd040b5ae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0099b0 tx=0x7fbcec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1782632684f9bb server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd01ffb700 1 -- 172.21.15.7:0/3163133894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00b860 con 0x7fbd04159380 2023-10-07T07:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd01ffb700 1 -- 172.21.15.7:0/3163133894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec00b9c0 con 0x7fbd04159380 2023-10-07T07:58:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.895+0000 7fbd01ffb700 1 -- 172.21.15.7:0/3163133894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00bd60 con 0x7fbd04159380 2023-10-07T07:58:04.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.896+0000 7fbd093d7700 1 -- 172.21.15.7:0/3163133894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd04159380 msgr2=0x7fbd040b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.896+0000 7fbd093d7700 1 --2- 172.21.15.7:0/3163133894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd04159380 0x7fbd040b5ae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0099b0 tx=0x7fbcec009250 comp rx=0 tx=0).stop 2023-10-07T07:58:04.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.896+0000 7fbd093d7700 1 -- 172.21.15.7:0/3163133894 shutdown_connections 2023-10-07T07:58:04.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.896+0000 7fbd093d7700 1 --2- 172.21.15.7:0/3163133894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd04159380 0x7fbd040b5ae0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.896+0000 7fbd093d7700 1 -- 172.21.15.7:0/3163133894 >> 172.21.15.7:0/3163133894 conn(0x7fbd04067be0 msgr2=0x7fbd04067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:04.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.896+0000 7fbd093d7700 1 -- 172.21.15.7:0/3163133894 shutdown_connections 2023-10-07T07:58:04.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.896+0000 7fbd093d7700 1 -- 172.21.15.7:0/3163133894 wait complete. 2023-10-07T07:58:04.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.897+0000 7fbd093d7700 1 Processor -- start 2023-10-07T07:58:04.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.897+0000 7fbd093d7700 1 -- start start 2023-10-07T07:58:04.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.898+0000 7fbd093d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd040b4270 0x7fbd0414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.898+0000 7fbd093d7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd040d0250 0x7fbd0415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.898+0000 7fbd093d7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd04159380 0x7fbd040b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.898+0000 7fbd093d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd041580c0 con 0x7fbd040b4270 2023-10-07T07:58:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.898+0000 7fbd093d7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0414c730 con 0x7fbd040d0250 2023-10-07T07:58:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.898+0000 7fbd093d7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd04157820 con 0x7fbd04159380 2023-10-07T07:58:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.898+0000 7fbd02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd040b4270 0x7fbd0414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.898+0000 7fbd02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd040b4270 0x7fbd0414c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48126/0 (socket says 172.21.15.7:48126) 2023-10-07T07:58:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.898+0000 7fbd02ffd700 1 -- 172.21.15.7:0/2748067725 learned_addr learned my addr 172.21.15.7:0/2748067725 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd037fe700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd04159380 0x7fbd040b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd027fc700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd040d0250 0x7fbd0415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd02ffd700 1 -- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd04159380 msgr2=0x7fbd040b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd02ffd700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd04159380 0x7fbd040b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd02ffd700 1 -- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd040d0250 msgr2=0x7fbd0415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:04.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd02ffd700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd040d0250 0x7fbd0415c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:04.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd02ffd700 1 -- 172.21.15.7:0/2748067725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0094f0 con 0x7fbd040b4270 2023-10-07T07:58:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd02ffd700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd040b4270 0x7fbd0414c1f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbd0406f850 tx=0x7fbcec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbcfaffd700 1 -- 172.21.15.7:0/2748067725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec01f070 con 0x7fbd040b4270 2023-10-07T07:58:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbcfaffd700 1 -- 172.21.15.7:0/2748067725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec011d10 con 0x7fbd040b4270 2023-10-07T07:58:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd04157b00 con 0x7fbd040b4270 2023-10-07T07:58:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbcfaffd700 1 -- 172.21.15.7:0/2748067725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec025670 con 0x7fbd040b4270 2023-10-07T07:58:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.899+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0414d980 con 0x7fbd040b4270 2023-10-07T07:58:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.900+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0414d7e0 con 0x7fbd040b4270 2023-10-07T07:58:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.901+0000 7fbcfaffd700 1 -- 172.21.15.7:0/2748067725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbcec023070 con 0x7fbd040b4270 2023-10-07T07:58:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.901+0000 7fbcfaffd700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbce4066f30 0x7fbce40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.901+0000 7fbcfaffd700 1 -- 172.21.15.7:0/2748067725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbcec086150 con 0x7fbd040b4270 2023-10-07T07:58:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.901+0000 7fbd027fc700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbce4066f30 0x7fbce40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.902+0000 7fbd027fc700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbce4066f30 0x7fbce40693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40011e0 tx=0x7fbcf4007bd0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:04.905+0000 7fbcfaffd700 1 -- 172.21.15.7:0/2748067725 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcec0534f0 con 0x7fbd040b4270 2023-10-07T07:58:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.154+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd0414d7e0 con 0x7fbd040b4270 2023-10-07T07:58:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.155+0000 7fbcfaffd700 1 -- 172.21.15.7:0/2748067725 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fbd0414d7e0 con 0x7fbd040b4270 2023-10-07T07:58:05.156 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:05.156 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2318752844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2318752844},{"type":"v1","addr":"172.21.15.172:6836","nonce":2318752844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1231430544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1231430544},{"type":"v1","addr":"172.21.15.172:6837","nonce":1231430544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1421504798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1421504798},{"type":"v1","addr":"172.21.15.7:6833","nonce":1421504798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:57:50.139330+0000","modified":"2023-10-07T07:57:53.727131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T07:58:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbce4066f30 msgr2=0x7fbce40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbce4066f30 0x7fbce40693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40011e0 tx=0x7fbcf4007bd0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd040b4270 msgr2=0x7fbd0414c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd040b4270 0x7fbd0414c1f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbd0406f850 tx=0x7fbcec017000 comp rx=0 tx=0).stop 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 shutdown_connections 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbce4066f30 0x7fbce40693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd04159380 0x7fbd040b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd040b4270 0x7fbd0414c1f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 --2- 172.21.15.7:0/2748067725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd040d0250 0x7fbd0415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 >> 172.21.15.7:0/2748067725 conn(0x7fbd04067be0 msgr2=0x7fbd0414cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 shutdown_connections 2023-10-07T07:58:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.157+0000 7fbd093d7700 1 -- 172.21.15.7:0/2748067725 wait complete. 2023-10-07T07:58:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2023-10-07T07:58:05.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-07T07:57:50.139330+0000', 'modified': '2023-10-07T07:57:53.727131+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/1878634661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 1878634661}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 1878634661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-07T07:58:05.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-07T07:58:05.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-07T07:58:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.716+0000 7fabf4442700 1 Processor -- start 2023-10-07T07:58:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.716+0000 7fabf4442700 1 -- start start 2023-10-07T07:58:05.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf4442700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec155f20 0x7fabec0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf4442700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabec1562f0 0x7fabec06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf4442700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec06b700 con 0x7fabec06e2c0 2023-10-07T07:58:05.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf4442700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec06aaf0 con 0x7fabec1562f0 2023-10-07T07:58:05.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf4442700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec06add0 con 0x7fabec155f20 2023-10-07T07:58:05.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf19dd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabec1562f0 0x7fabec06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf19dd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabec1562f0 0x7fabec06a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48134/0 (socket says 172.21.15.7:48134) 2023-10-07T07:58:05.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf29df700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fabec06e2c0 0x7fabec159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53106/0 (socket says 172.21.15.7:53106) 2023-10-07T07:58:05.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf19dd700 1 -- 172.21.15.7:0/533047523 learned_addr learned my addr 172.21.15.7:0/533047523 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:05.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.717+0000 7fabf21de700 1 --2- 172.21.15.7:0/533047523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec155f20 0x7fabec0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf19dd700 1 -- 172.21.15.7:0/533047523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec155f20 msgr2=0x7fabec0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:05.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf19dd700 1 --2- 172.21.15.7:0/533047523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec155f20 0x7fabec0c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf19dd700 1 -- 172.21.15.7:0/533047523 >> v1:172.21.15.7:6789/0 conn(0x7fabec06e2c0 legacy=0x7fabec159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf19dd700 1 -- 172.21.15.7:0/533047523 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec06b0b0 con 0x7fabec1562f0 2023-10-07T07:58:05.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf11dc700 1 -- 172.21.15.7:0/533047523 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130091098 0 0) 0x7fabec06b700 con 0x7fabec06e2c0 2023-10-07T07:58:05.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf19dd700 1 --2- 172.21.15.7:0/533047523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1562f0 0x7fabec06a3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0099b0 tx=0x7fabdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ce77e9930df0c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:05.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf11dc700 1 -- 172.21.15.7:0/533047523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc009ea0 con 0x7fabec1562f0 2023-10-07T07:58:05.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf11dc700 1 -- 172.21.15.7:0/533047523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc00b860 con 0x7fabec1562f0 2023-10-07T07:58:05.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf11dc700 1 -- 172.21.15.7:0/533047523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00bc00 con 0x7fabec1562f0 2023-10-07T07:58:05.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.718+0000 7fabf4442700 1 -- 172.21.15.7:0/533047523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1562f0 msgr2=0x7fabec06a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.719+0000 7fabf4442700 1 --2- 172.21.15.7:0/533047523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1562f0 0x7fabec06a3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0099b0 tx=0x7fabdc009250 comp rx=0 tx=0).stop 2023-10-07T07:58:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.719+0000 7fabf4442700 1 -- 172.21.15.7:0/533047523 shutdown_connections 2023-10-07T07:58:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.719+0000 7fabf4442700 1 --2- 172.21.15.7:0/533047523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec155f20 0x7fabec0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.719+0000 7fabf4442700 1 --2- 172.21.15.7:0/533047523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1562f0 0x7fabec06a3f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.719+0000 7fabf4442700 1 -- 172.21.15.7:0/533047523 >> 172.21.15.7:0/533047523 conn(0x7fabec068bd0 msgr2=0x7fabec0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.719+0000 7fabf4442700 1 -- 172.21.15.7:0/533047523 shutdown_connections 2023-10-07T07:58:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.719+0000 7fabf4442700 1 -- 172.21.15.7:0/533047523 wait complete. 2023-10-07T07:58:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.720+0000 7fabf4442700 1 Processor -- start 2023-10-07T07:58:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.720+0000 7fabf4442700 1 -- start start 2023-10-07T07:58:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf4442700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabec06e2c0 0x7fabec0c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf4442700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec155f20 0x7fabec0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf4442700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec1562f0 0x7fabec14c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf4442700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0afd50 con 0x7fabec155f20 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf4442700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0c45a0 con 0x7fabec06e2c0 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf4442700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0c4840 con 0x7fabec1562f0 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf19dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec155f20 0x7fabec0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf19dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec155f20 0x7fabec0cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48146/0 (socket says 172.21.15.7:48146) 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf19dd700 1 -- 172.21.15.7:0/3400868440 learned_addr learned my addr 172.21.15.7:0/3400868440 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf21de700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabec06e2c0 0x7fabec0c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf29df700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec1562f0 0x7fabec14c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf19dd700 1 -- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec1562f0 msgr2=0x7fabec14c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf19dd700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec1562f0 0x7fabec14c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf19dd700 1 -- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabec06e2c0 msgr2=0x7fabec0c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf19dd700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabec06e2c0 0x7fabec0c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf19dd700 1 -- 172.21.15.7:0/3400868440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0094f0 con 0x7fabec155f20 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.721+0000 7fabf19dd700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec155f20 0x7fabec0cb6a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fabec069020 tx=0x7fabdc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.722+0000 7fabe27fc700 1 -- 172.21.15.7:0/3400868440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc000d10 con 0x7fabec155f20 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.722+0000 7fabe27fc700 1 -- 172.21.15.7:0/3400868440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc00bbe0 con 0x7fabec155f20 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.722+0000 7fabe27fc700 1 -- 172.21.15.7:0/3400868440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc011690 con 0x7fabec155f20 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.722+0000 7fabf4442700 1 -- 172.21.15.7:0/3400868440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec0c4b20 con 0x7fabec155f20 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.722+0000 7fabf4442700 1 -- 172.21.15.7:0/3400868440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec0b0280 con 0x7fabec155f20 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.723+0000 7fabe27fc700 1 -- 172.21.15.7:0/3400868440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabdc011ac0 con 0x7fabec155f20 2023-10-07T07:58:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.724+0000 7fabe27fc700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd0066f30 0x7fabd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.724+0000 7fabe27fc700 1 -- 172.21.15.7:0/3400868440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fabdc085ce0 con 0x7fabec155f20 2023-10-07T07:58:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.724+0000 7fabf21de700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd0066f30 0x7fabd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.724+0000 7fabf21de700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd0066f30 0x7fabd00693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009db0 tx=0x7fabe80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.726+0000 7fabfd756b80 1 Processor -- start 2023-10-07T07:58:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.726+0000 7fabfd756b80 1 -- start start 2023-10-07T07:58:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb188e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb191680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb192010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabfd756b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb05b310 con 0x55bccb191bc0 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabfd756b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb098130 con 0x55bccb188a40 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabfd756b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bccb095af0 con 0x55bccb16e530 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf29df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb192010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf29df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb192010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48160/0 (socket says 172.21.15.7:48160) 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf29df700 1 -- 172.21.15.7:0/2347212354 learned_addr learned my addr 172.21.15.7:0/2347212354 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf19dd700 1 --2- 172.21.15.7:0/2347212354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb191680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf21de700 1 --2- 172.21.15.7:0/2347212354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb188e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf29df700 1 -- 172.21.15.7:0/2347212354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 msgr2=0x55bccb191680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf29df700 1 --2- 172.21.15.7:0/2347212354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb191680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf29df700 1 -- 172.21.15.7:0/2347212354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 msgr2=0x55bccb188e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf29df700 1 --2- 172.21.15.7:0/2347212354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb188e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.727+0000 7fabf29df700 1 -- 172.21.15.7:0/2347212354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bccaf840a0 con 0x55bccb191bc0 2023-10-07T07:58:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.728+0000 7fabf29df700 1 --2- 172.21.15.7:0/2347212354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb192010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fabe40020e0 tx=0x7fabe4001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a013c8c665824e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.728+0000 7fabcf7fe700 1 -- 172.21.15.7:0/2347212354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4004d80 con 0x55bccb191bc0 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.728+0000 7fabcf7fe700 1 -- 172.21.15.7:0/2347212354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40079e0 con 0x55bccb191bc0 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.728+0000 7fabcf7fe700 1 -- 172.21.15.7:0/2347212354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4013600 con 0x55bccb191bc0 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 -- 172.21.15.7:0/2347212354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 msgr2=0x55bccb192010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 --2- 172.21.15.7:0/2347212354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb192010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fabe40020e0 tx=0x7fabe4001a60 comp rx=0 tx=0).stop 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 -- 172.21.15.7:0/2347212354 shutdown_connections 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 --2- 172.21.15.7:0/2347212354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb192010 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 --2- 172.21.15.7:0/2347212354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb188e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 --2- 172.21.15.7:0/2347212354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb191680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 -- 172.21.15.7:0/2347212354 >> 172.21.15.7:0/2347212354 conn(0x55bccb18be70 msgr2=0x55bccb18c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 -- 172.21.15.7:0/2347212354 shutdown_connections 2023-10-07T07:58:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 -- 172.21.15.7:0/2347212354 wait complete. 2023-10-07T07:58:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 -- start start 2023-10-07T07:58:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.729+0000 7fabfd756b80 1 Processor -- start 2023-10-07T07:58:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.730+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb188180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.730+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb16c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.730+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb16ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.730+0000 7fabfd756b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb0918e0 con 0x55bccb191bc0 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.730+0000 7fabfd756b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb03d980 con 0x55bccb188a40 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.730+0000 7fabfd756b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bccb03d620 con 0x55bccb16e530 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.730+0000 7fabfd756b80 10 client.0 _ll_register_callbacks cb 0x55bccaea0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf29df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb16ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf19dd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb16c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf29df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb16ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48174/0 (socket says 172.21.15.7:48174) 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf29df700 1 -- 172.21.15.7:0/3586840057 learned_addr learned my addr 172.21.15.7:0/3586840057 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf21de700 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb188180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf19dd700 1 -- 172.21.15.7:0/3586840057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 msgr2=0x55bccb188180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf19dd700 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb188180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf19dd700 1 -- 172.21.15.7:0/3586840057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 msgr2=0x55bccb16ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf19dd700 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb16ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf19dd700 1 -- 172.21.15.7:0/3586840057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe4001d40 con 0x55bccb188a40 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabf19dd700 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb16c4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0842e0 tx=0x7fabdc084160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b150cce39d06be server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.731+0000 7fabccff9700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabccff9700 1 -- 172.21.15.7:0/3586840057 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc06d070 con 0x55bccb188a40 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabccff9700 1 -- 172.21.15.7:0/3586840057 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fabdc056c10 con 0x55bccb188a40 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabfd756b80 1 -- 172.21.15.7:0/3586840057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bccb136560 con 0x55bccb188a40 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabccff9700 1 -- 172.21.15.7:0/3586840057 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabdc059e60 con 0x55bccb188a40 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabfd756b80 10 client.4588 fetch_fsmap learned FSMap version 8 2023-10-07T07:58:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabfd756b80 1 -- 172.21.15.7:0/3586840057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55bccb0603c0 con 0x55bccb188a40 2023-10-07T07:58:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabccff9700 1 -- 172.21.15.7:0/3586840057 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bccb136560 con 0x55bccb188a40 2023-10-07T07:58:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabccff9700 1 -- 172.21.15.7:0/3586840057 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fabdc05d720 con 0x55bccb188a40 2023-10-07T07:58:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabfd756b80 10 client.4588 fetch_fsmap finished waiting for FSMap version 8 2023-10-07T07:58:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabfd756b80 10 client.4588 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-07T07:58:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabfd756b80 20 client.4588 populate_metadata read hostname 'smithi007' 2023-10-07T07:58:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabfd756b80 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] conn(0x55bccb190760 0x55bccb153d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabfd756b80 4 client.4588 mds_command: new command op to 4263 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T07:58:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabfd756b80 1 -- 172.21.15.7:0/3586840057 --> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bccb1130e0 con 0x55bccb190760 2023-10-07T07:58:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.732+0000 7fabf21de700 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] conn(0x55bccb190760 0x55bccb153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.733+0000 7fabccff9700 10 client.4588 ms_handle_connect on v2:172.21.15.7:6834/1878634661 2023-10-07T07:58:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.733+0000 7fabf21de700 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] conn(0x55bccb190760 0x55bccb153d80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df7e35b4affa9b0f server_cookie=6040bfa5f4cac150 in_seq=0 out_seq=0 2023-10-07T07:58:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.733+0000 7fabccff9700 1 -- 172.21.15.7:0/3586840057 <== mds.0 v2:172.21.15.7:6834/1878634661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55bccb1130e0 con 0x55bccb190760 2023-10-07T07:58:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.733+0000 7fabccff9700 10 client.4588 handle_command_reply: tid=0 2023-10-07T07:58:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.733+0000 7fabfd756b80 1 client.4588 shutdown 2023-10-07T07:58:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.733+0000 7fabfd756b80 1 -- 172.21.15.7:0/3586840057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 msgr2=0x55bccb16c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb16c4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0842e0 tx=0x7fabdc084160 comp rx=0 tx=0).stop 2023-10-07T07:58:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 1 -- 172.21.15.7:0/3586840057 shutdown_connections 2023-10-07T07:58:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] conn(0x55bccb190760 0x55bccb153d80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb16ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb188180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 1 --2- 172.21.15.7:0/3586840057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb16c4f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabccff9700 0 client.4588 ms_handle_reset on v2:172.21.15.7:6834/1878634661 2023-10-07T07:58:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 1 -- 172.21.15.7:0/3586840057 >> 172.21.15.7:0/3586840057 conn(0x55bccb18be70 msgr2=0x55bccb13e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 1 -- 172.21.15.7:0/3586840057 shutdown_connections 2023-10-07T07:58:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 1 -- 172.21.15.7:0/3586840057 wait complete. 2023-10-07T07:58:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.734+0000 7fabfd756b80 20 client.4588 trim_cache size 0 max 16384 2023-10-07T07:58:05.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.752+0000 7fabfd756b80 1 Processor -- start 2023-10-07T07:58:05.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.752+0000 7fabfd756b80 1 -- start start 2023-10-07T07:58:05.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb142b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb1430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb18d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabfd756b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccaf291e0 con 0x55bccb191bc0 2023-10-07T07:58:05.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabfd756b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb16df40 con 0x55bccb188a40 2023-10-07T07:58:05.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabfd756b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bccb144510 con 0x55bccb16e530 2023-10-07T07:58:05.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf29df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb18d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf29df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb18d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48184/0 (socket says 172.21.15.7:48184) 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf29df700 1 -- 172.21.15.7:0/1904160370 learned_addr learned my addr 172.21.15.7:0/1904160370 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf19dd700 1 --2- 172.21.15.7:0/1904160370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb1430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf21de700 1 --2- 172.21.15.7:0/1904160370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf29df700 1 -- 172.21.15.7:0/1904160370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 msgr2=0x55bccb142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf29df700 1 --2- 172.21.15.7:0/1904160370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf29df700 1 -- 172.21.15.7:0/1904160370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 msgr2=0x55bccb1430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf29df700 1 --2- 172.21.15.7:0/1904160370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb1430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf29df700 1 -- 172.21.15.7:0/1904160370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bccb152f80 con 0x55bccb191bc0 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.753+0000 7fabf29df700 1 --2- 172.21.15.7:0/1904160370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb18d2e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fabe4001cd0 tx=0x7fabe4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2e23dedfb23fa server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabcffff700 1 -- 172.21.15.7:0/1904160370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4001790 con 0x55bccb191bc0 2023-10-07T07:58:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabcffff700 1 -- 172.21.15.7:0/1904160370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe4004d60 con 0x55bccb191bc0 2023-10-07T07:58:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabcffff700 1 -- 172.21.15.7:0/1904160370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40133f0 con 0x55bccb191bc0 2023-10-07T07:58:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabfd756b80 1 -- 172.21.15.7:0/1904160370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 msgr2=0x55bccb18d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1904160370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb18d2e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fabe4001cd0 tx=0x7fabe4018000 comp rx=0 tx=0).stop 2023-10-07T07:58:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabfd756b80 1 -- 172.21.15.7:0/1904160370 shutdown_connections 2023-10-07T07:58:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1904160370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb18d2e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1904160370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb16e530 0x55bccb142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1904160370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb188a40 0x55bccb1430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabfd756b80 1 -- 172.21.15.7:0/1904160370 >> 172.21.15.7:0/1904160370 conn(0x55bccb18be70 msgr2=0x55bccb1514f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabfd756b80 1 -- 172.21.15.7:0/1904160370 shutdown_connections 2023-10-07T07:58:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.754+0000 7fabfd756b80 1 -- 172.21.15.7:0/1904160370 wait complete. 2023-10-07T07:58:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.755+0000 7fabfd756b80 1 -- start start 2023-10-07T07:58:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.755+0000 7fabfd756b80 1 Processor -- start 2023-10-07T07:58:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb16e530 0x55bccb1a1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb188a40 0x55bccb1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabfd756b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb1ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabfd756b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb16c4f0 con 0x55bccb191bc0 2023-10-07T07:58:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabfd756b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bccb16cd90 con 0x55bccb16e530 2023-10-07T07:58:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabfd756b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bccb13de20 con 0x55bccb188a40 2023-10-07T07:58:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabfd756b80 10 client.0 _ll_register_callbacks cb 0x55bccaea0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T07:58:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabf29df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb1ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabf29df700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb1ed000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48200/0 (socket says 172.21.15.7:48200) 2023-10-07T07:58:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabf29df700 1 -- 172.21.15.7:0/1452450692 learned_addr learned my addr 172.21.15.7:0/1452450692 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabf21de700 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb16e530 0x55bccb1a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.756+0000 7fabf19dd700 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb188a40 0x55bccb1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabf29df700 1 -- 172.21.15.7:0/1452450692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb188a40 msgr2=0x55bccb1a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabf29df700 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb188a40 0x55bccb1a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabf29df700 1 -- 172.21.15.7:0/1452450692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb16e530 msgr2=0x55bccb1a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabf29df700 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb16e530 0x55bccb1a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabf29df700 1 -- 172.21.15.7:0/1452450692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bccaf840a0 con 0x55bccb191bc0 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabf21de700 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb16e530 0x55bccb1a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabf29df700 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb1ed000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007bd0 tx=0x7fabe4023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed14e8563af2f02e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabccff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabccff9700 1 -- 172.21.15.7:0/1452450692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe402c450 con 0x55bccb191bc0 2023-10-07T07:58:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabccff9700 1 -- 172.21.15.7:0/1452450692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fabe402c5b0 con 0x55bccb191bc0 2023-10-07T07:58:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabfd756b80 1 -- 172.21.15.7:0/1452450692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabe40079e0 con 0x55bccb191bc0 2023-10-07T07:58:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabccff9700 1 -- 172.21.15.7:0/1452450692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabe402c980 con 0x55bccb191bc0 2023-10-07T07:58:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabfd756b80 10 client.4653 fetch_fsmap learned FSMap version 8 2023-10-07T07:58:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabfd756b80 1 -- 172.21.15.7:0/1452450692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55bccaef81d0 con 0x55bccb191bc0 2023-10-07T07:58:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabccff9700 1 -- 172.21.15.7:0/1452450692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40079e0 con 0x55bccb191bc0 2023-10-07T07:58:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabccff9700 1 -- 172.21.15.7:0/1452450692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fabe4012070 con 0x55bccb191bc0 2023-10-07T07:58:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabfd756b80 10 client.4653 fetch_fsmap finished waiting for FSMap version 8 2023-10-07T07:58:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabfd756b80 10 client.4653 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-07T07:58:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabfd756b80 20 client.4653 populate_metadata read hostname 'smithi007' 2023-10-07T07:58:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] conn(0x55bccb18c8c0 0x55bccb1ed540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabfd756b80 4 client.4653 mds_command: new command op to 4263 tid=0[{"prefix": "status"}] 2023-10-07T07:58:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.757+0000 7fabfd756b80 1 -- 172.21.15.7:0/1452450692 --> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bccb0348d0 con 0x55bccb18c8c0 2023-10-07T07:58:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabf21de700 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] conn(0x55bccb18c8c0 0x55bccb1ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabccff9700 10 client.4653 ms_handle_connect on v2:172.21.15.7:6834/1878634661 2023-10-07T07:58:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabf21de700 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] conn(0x55bccb18c8c0 0x55bccb1ed540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee9df2f7cd2305e server_cookie=217a03b19d298b72 in_seq=0 out_seq=0 2023-10-07T07:58:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabccff9700 1 -- 172.21.15.7:0/1452450692 <== mds.0 v2:172.21.15.7:6834/1878634661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55bccb0348d0 con 0x55bccb18c8c0 2023-10-07T07:58:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabccff9700 10 client.4653 handle_command_reply: tid=0 2023-10-07T07:58:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabfd756b80 1 client.4653 shutdown 2023-10-07T07:58:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabfd756b80 1 -- 172.21.15.7:0/1452450692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 msgr2=0x55bccb1ed000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb1ed000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007bd0 tx=0x7fabe4023040 comp rx=0 tx=0).stop 2023-10-07T07:58:05.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabfd756b80 1 -- 172.21.15.7:0/1452450692 shutdown_connections 2023-10-07T07:58:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.7:6834/1878634661,v1:172.21.15.7:6835/1878634661] conn(0x55bccb18c8c0 0x55bccb1ed540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bccb191bc0 0x55bccb1ed000 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bccb16e530 0x55bccb1a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabfd756b80 1 --2- 172.21.15.7:0/1452450692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bccb188a40 0x55bccb1a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabccff9700 0 client.4653 ms_handle_reset on v2:172.21.15.7:6834/1878634661 2023-10-07T07:58:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.758+0000 7fabfd756b80 1 -- 172.21.15.7:0/1452450692 >> 172.21.15.7:0/1452450692 conn(0x55bccb18be70 msgr2=0x55bccb13f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabfd756b80 1 -- 172.21.15.7:0/1452450692 shutdown_connections 2023-10-07T07:58:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabfd756b80 1 -- 172.21.15.7:0/1452450692 wait complete. 2023-10-07T07:58:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabfd756b80 20 client.4653 trim_cache size 0 max 16384 2023-10-07T07:58:05.770 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:58:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "c71d63e6-6de9-4139-83e7-b0bb7f6375d7", 2023-10-07T07:58:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2023-10-07T07:58:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4263, 2023-10-07T07:58:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2023-10-07T07:58:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2023-10-07T07:58:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2023-10-07T07:58:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 15.043156772, 2023-10-07T07:58:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 8, 2023-10-07T07:58:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 21, 2023-10-07T07:58:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 21, 2023-10-07T07:58:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 22.145029873999999 2023-10-07T07:58:05.772 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:58:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 -- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd0066f30 msgr2=0x7fabd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd0066f30 0x7fabd00693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009db0 tx=0x7fabe80091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 -- 172.21.15.7:0/3400868440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec155f20 msgr2=0x7fabec0cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec155f20 0x7fabec0cb6a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fabec069020 tx=0x7fabdc017000 comp rx=0 tx=0).stop 2023-10-07T07:58:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 -- 172.21.15.7:0/3400868440 shutdown_connections 2023-10-07T07:58:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd0066f30 0x7fabd00693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabec1562f0 0x7fabec14c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabec06e2c0 0x7fabec0c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 --2- 172.21.15.7:0/3400868440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec155f20 0x7fabec0cb6a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 -- 172.21.15.7:0/3400868440 >> 172.21.15.7:0/3400868440 conn(0x7fabec068bd0 msgr2=0x7fabec0c22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 -- 172.21.15.7:0/3400868440 shutdown_connections 2023-10-07T07:58:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:05.759+0000 7fabf4442700 1 -- 172.21.15.7:0/3400868440 wait complete. 2023-10-07T07:58:05.777 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-07T07:58:05.800 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-07T07:58:05.801 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-07T07:58:05.801 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-07T07:58:05.801 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-07T07:58:05.802 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-07T07:58:05.802 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-07T07:58:05.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:05.802 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-07T07:58:05.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:05.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2023-10-07T07:58:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2023-10-07T07:58:05.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T07:58:05.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:05.904 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-10-07T07:58:05.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:05.927 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2023-10-07T07:58:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2023-10-07T07:58:06.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T07:58:06.004 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-07T07:58:06.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T07:58:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.143+0000 7f2ce40ce700 1 Processor -- start 2023-10-07T07:58:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.143+0000 7f2ce40ce700 1 -- start start 2023-10-07T07:58:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.144+0000 7f2ce40ce700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2cdc06f280 0x7f2cdc161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.144+0000 7f2ce40ce700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc0b4760 0x7f2cdc15d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.144+0000 7f2ce40ce700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc0b57e0 con 0x7f2cdc06eeb0 2023-10-07T07:58:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.144+0000 7f2ce40ce700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc0ccd20 con 0x7f2cdc06f280 2023-10-07T07:58:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.144+0000 7f2ce40ce700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0b4f40 con 0x7f2cdc0b4760 2023-10-07T07:58:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce1669700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2cdc06f280 0x7f2cdc161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce266b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc0b4760 0x7f2cdc15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce1e6a700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2cdc06eeb0 0x7f2cdc0cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:46160/0 (socket says 172.21.15.7:46160) 2023-10-07T07:58:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce1e6a700 1 -- 172.21.15.7:0/3237055992 learned_addr learned my addr 172.21.15.7:0/3237055992 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce1669700 1 -- 172.21.15.7:0/3237055992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc0b4760 msgr2=0x7f2cdc15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce1669700 1 --2- 172.21.15.7:0/3237055992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc0b4760 0x7f2cdc15d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce1669700 1 -- 172.21.15.7:0/3237055992 >> v1:172.21.15.172:6789/0 conn(0x7f2cdc06eeb0 legacy=0x7f2cdc0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce1669700 1 -- 172.21.15.7:0/3237055992 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0b5220 con 0x7f2cdc06f280 2023-10-07T07:58:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce1669700 1 --2- 172.21.15.7:0/3237055992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cdc06f280 0x7f2cdc161f50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80098e0 tx=0x7f2cd80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=835d7bf0cf74374f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce0e68700 1 -- 172.21.15.7:0/3237055992 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800b640 con 0x7f2cdc06f280 2023-10-07T07:58:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce0e68700 1 -- 172.21.15.7:0/3237055992 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd800b7a0 con 0x7f2cdc06f280 2023-10-07T07:58:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.145+0000 7f2ce0e68700 1 -- 172.21.15.7:0/3237055992 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800bb40 con 0x7f2cdc06f280 2023-10-07T07:58:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.146+0000 7f2ce40ce700 1 -- 172.21.15.7:0/3237055992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cdc06f280 msgr2=0x7f2cdc161f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.146+0000 7f2ce40ce700 1 --2- 172.21.15.7:0/3237055992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cdc06f280 0x7f2cdc161f50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80098e0 tx=0x7f2cd80091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.146+0000 7f2ce40ce700 1 -- 172.21.15.7:0/3237055992 shutdown_connections 2023-10-07T07:58:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.146+0000 7f2ce40ce700 1 --2- 172.21.15.7:0/3237055992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc0b4760 0x7f2cdc15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.146+0000 7f2ce40ce700 1 --2- 172.21.15.7:0/3237055992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cdc06f280 0x7f2cdc161f50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.146+0000 7f2ce40ce700 1 -- 172.21.15.7:0/3237055992 >> 172.21.15.7:0/3237055992 conn(0x7f2cdc067be0 msgr2=0x7f2cdc067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.146+0000 7f2ce40ce700 1 -- 172.21.15.7:0/3237055992 shutdown_connections 2023-10-07T07:58:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.146+0000 7f2ce40ce700 1 -- 172.21.15.7:0/3237055992 wait complete. 2023-10-07T07:58:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.147+0000 7f2ce40ce700 1 Processor -- start 2023-10-07T07:58:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.147+0000 7f2ce40ce700 1 -- start start 2023-10-07T07:58:06.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce40ce700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cdc06eeb0 0x7f2cdc155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:06.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce40ce700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc06f280 0x7f2cdc06c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:06.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce40ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cdc0b4760 0x7f2cdc067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:06.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce40ce700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc16dbc0 con 0x7f2cdc0b4760 2023-10-07T07:58:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce40ce700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc06ca60 con 0x7f2cdc06eeb0 2023-10-07T07:58:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce40ce700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc06cd40 con 0x7f2cdc06f280 2023-10-07T07:58:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce266b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cdc0b4760 0x7f2cdc067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce266b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cdc0b4760 0x7f2cdc067fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48210/0 (socket says 172.21.15.7:48210) 2023-10-07T07:58:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce266b700 1 -- 172.21.15.7:0/1093682602 learned_addr learned my addr 172.21.15.7:0/1093682602 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce1e6a700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cdc06eeb0 0x7f2cdc155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.148+0000 7f2ce1669700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc06f280 0x7f2cdc06c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ce266b700 1 -- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc06f280 msgr2=0x7f2cdc06c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ce266b700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc06f280 0x7f2cdc06c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ce266b700 1 -- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cdc06eeb0 msgr2=0x7f2cdc155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ce266b700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cdc06eeb0 0x7f2cdc155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ce266b700 1 -- 172.21.15.7:0/1093682602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8009420 con 0x7f2cdc0b4760 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ce266b700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cdc0b4760 0x7f2cdc067fb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009df0 tx=0x7f2cd400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ccdffb700 1 -- 172.21.15.7:0/1093682602 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400caa0 con 0x7f2cdc0b4760 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ccdffb700 1 -- 172.21.15.7:0/1093682602 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd400f040 con 0x7f2cdc0b4760 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc06d020 con 0x7f2cdc0b4760 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ccdffb700 1 -- 172.21.15.7:0/1093682602 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400ce70 con 0x7f2cdc0b4760 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.149+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc156380 con 0x7f2cdc0b4760 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.150+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc06d540 con 0x7f2cdc0b4760 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.151+0000 7f2ccdffb700 1 -- 172.21.15.7:0/1093682602 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2cd4014690 con 0x7f2cdc0b4760 2023-10-07T07:58:06.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.151+0000 7f2ccdffb700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2cc0066f30 0x7f2cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.151+0000 7f2ccdffb700 1 -- 172.21.15.7:0/1093682602 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2cd4084d40 con 0x7f2cdc0b4760 2023-10-07T07:58:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.152+0000 7f2ce1e6a700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2cc0066f30 0x7f2cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.152+0000 7f2ce1e6a700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2cc0066f30 0x7f2cc00693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0007130 tx=0x7f2cd0007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.155+0000 7f2ccdffb700 1 -- 172.21.15.7:0/1093682602 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cdc06d540 con 0x7f2cdc0b4760 2023-10-07T07:58:06.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.406+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2cdc06d540 con 0x7f2cdc0b4760 2023-10-07T07:58:06.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.406+0000 7f2ccdffb700 1 -- 172.21.15.7:0/1093682602 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2cd4018060 con 0x7f2cdc0b4760 2023-10-07T07:58:06.407 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T07:58:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2cc0066f30 msgr2=0x7f2cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2cc0066f30 0x7f2cc00693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0007130 tx=0x7f2cd0007eb0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cdc0b4760 msgr2=0x7f2cdc067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cdc0b4760 0x7f2cdc067fb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009df0 tx=0x7f2cd400b040 comp rx=0 tx=0).stop 2023-10-07T07:58:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 shutdown_connections 2023-10-07T07:58:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2cc0066f30 0x7f2cc00693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cdc0b4760 0x7f2cdc067fb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cdc06eeb0 0x7f2cdc155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 --2- 172.21.15.7:0/1093682602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2cdc06f280 0x7f2cdc06c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 >> 172.21.15.7:0/1093682602 conn(0x7f2cdc067be0 msgr2=0x7f2cdc162ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.408+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 shutdown_connections 2023-10-07T07:58:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:06.409+0000 7f2ce40ce700 1 -- 172.21.15.7:0/1093682602 wait complete. 2023-10-07T07:58:06.423 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T07:58:06.423 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T07:58:06.423 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-07T07:58:06.423 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-07T07:58:06.423 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:06.423 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T07:58:06.423 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T07:58:06.423 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T07:58:06.423 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T07:58:06.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:06.424 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-07T07:58:06.444 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:06.444 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2023-10-07T07:58:06.445 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T07:58:06.446 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:06.446 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2023-10-07T07:58:06.446 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-07T07:58:06.446 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42448sec preferred_lft 42448sec 2023-10-07T07:58:06.446 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-07T07:58:06.446 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:06.446 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:06.446 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:06.446 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2023-10-07T07:58:06.447 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-07T07:58:06.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:06.447 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:06.447 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:06.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2023-10-07T07:58:06.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2023-10-07T07:58:06.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2023-10-07T07:58:06.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-07T07:58:06.448 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:06.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:06.630 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T07:58:06.668 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-07T07:58:06.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:06.670 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-07T07:58:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-07T07:58:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T07:58:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-07T07:58:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-07T07:58:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T07:58:06.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:06.755 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:06.755 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:06.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T07:58:06.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T07:58:06.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T07:58:06.755 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:07.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:07.055 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-07T07:58:07.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:07.078 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2023-10-07T07:58:07.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:07.142 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:07.142 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:07.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T07:58:07.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-07T07:58:07.142 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:07.254 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-07T07:58:07.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:07.254 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:07.254 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:07.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-07T07:58:07.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-07T07:58:07.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-07T07:58:07.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-07T07:58:07.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-07T07:58:07.255 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:07.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:07.480 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:07.480 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:07.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2023-10-07T07:58:07.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2023-10-07T07:58:07.480 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:07.592 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-07T07:58:07.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:07.593 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:07.613 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-07T07:58:07.614 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:07.614 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:07.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T07:58:07.752 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:07.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:07.828 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:07.829 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:07.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:07.830 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T07:58:07.893 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T07:58:07.893 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-07T07:58:07.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T07:58:07.971 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:07.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:08.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-10-07T07:58:08.037+0000 7fe8e37db540 -1 init, newargv = 0x55b39b227bc0 newargc=15 2023-10-07T07:58:08.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[128861]: starting ceph client 2023-10-07T07:58:08.042 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:08.044 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:08.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:08.044 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T07:58:08.045 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[128861]: starting fuse 2023-10-07T07:58:08.107 INFO:teuthology.orchestra.run.smithi007.stdout:47 2023-10-07T07:58:08.107 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-07T07:58:08.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-07T07:58:08.108 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-07T07:58:08.109 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T07:58:08.110 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:08.110 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2023-10-07T07:58:08.110 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T07:58:08.246 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.128861.asok 2023-10-07T07:58:08.428 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.128861.asok 2023-10-07T07:58:08.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:08.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.128861.asok status 2023-10-07T07:58:08.604 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:58:08.604 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-07T07:58:08.604 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "128861", 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4662, 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-07T07:58:08.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4662 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1025369904 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T07:58:08.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1025369904 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4662 192.168.0.1:0/1025369904", 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/1025369904", 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 8, 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 21, 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-07T07:58:08.607 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:58:08.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T07:58:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.746+0000 7ff5b6035700 1 Processor -- start 2023-10-07T07:58:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.746+0000 7ff5b6035700 1 -- start start 2023-10-07T07:58:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.747+0000 7ff5b6035700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5b0155f20 0x7ff5b00c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.747+0000 7ff5b6035700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff5b01562f0 0x7ff5b006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.747+0000 7ff5b6035700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b006b700 con 0x7ff5b006e2c0 2023-10-07T07:58:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.747+0000 7ff5b6035700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b006aaf0 con 0x7ff5b0155f20 2023-10-07T07:58:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.747+0000 7ff5b6035700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b006add0 con 0x7ff5b01562f0 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.747+0000 7ff5af7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5b0155f20 0x7ff5b00c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.747+0000 7ff5affff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff5b006e2c0 0x7ff5b006e6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53110/0 (socket says 172.21.15.7:53110) 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.747+0000 7ff5affff700 1 -- 172.21.15.7:0/3869714220 learned_addr learned my addr 172.21.15.7:0/3869714220 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.747+0000 7ff5aeffd700 1 --2- 172.21.15.7:0/3869714220 >> v2:172.21.15.172:3300/0 conn(0x7ff5b01562f0 0x7ff5b006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.748+0000 7ff5af7fe700 1 -- 172.21.15.7:0/3869714220 >> v2:172.21.15.172:3300/0 conn(0x7ff5b01562f0 msgr2=0x7ff5b006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.748+0000 7ff5af7fe700 1 --2- 172.21.15.7:0/3869714220 >> v2:172.21.15.172:3300/0 conn(0x7ff5b01562f0 0x7ff5b006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.748+0000 7ff5af7fe700 1 -- 172.21.15.7:0/3869714220 >> v1:172.21.15.7:6789/0 conn(0x7ff5b006e2c0 legacy=0x7ff5b006e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.748+0000 7ff5ae7fc700 1 -- 172.21.15.7:0/3869714220 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888332659 0 0) 0x7ff5b006b700 con 0x7ff5b006e2c0 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.748+0000 7ff5af7fe700 1 -- 172.21.15.7:0/3869714220 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b006b0b0 con 0x7ff5b0155f20 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.748+0000 7ff5af7fe700 1 --2- 172.21.15.7:0/3869714220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 0x7ff5b00c92b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff5b006e0b0 tx=0x7ff5a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13317137593a75e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.748+0000 7ff5ae7fc700 1 -- 172.21.15.7:0/3869714220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000b6b0 con 0x7ff5b0155f20 2023-10-07T07:58:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.748+0000 7ff5ae7fc700 1 -- 172.21.15.7:0/3869714220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a000b810 con 0x7ff5b0155f20 2023-10-07T07:58:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.748+0000 7ff5ae7fc700 1 -- 172.21.15.7:0/3869714220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000bbb0 con 0x7ff5b0155f20 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.749+0000 7ff5b6035700 1 -- 172.21.15.7:0/3869714220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 msgr2=0x7ff5b00c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.749+0000 7ff5b6035700 1 --2- 172.21.15.7:0/3869714220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 0x7ff5b00c92b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff5b006e0b0 tx=0x7ff5a00091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.749+0000 7ff5b6035700 1 -- 172.21.15.7:0/3869714220 shutdown_connections 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.749+0000 7ff5b6035700 1 --2- 172.21.15.7:0/3869714220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 0x7ff5b00c92b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.749+0000 7ff5b6035700 1 --2- 172.21.15.7:0/3869714220 >> v2:172.21.15.172:3300/0 conn(0x7ff5b01562f0 0x7ff5b006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.749+0000 7ff5b6035700 1 -- 172.21.15.7:0/3869714220 >> 172.21.15.7:0/3869714220 conn(0x7ff5b0068bd0 msgr2=0x7ff5b00ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.749+0000 7ff5b6035700 1 -- 172.21.15.7:0/3869714220 shutdown_connections 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.749+0000 7ff5b6035700 1 -- 172.21.15.7:0/3869714220 wait complete. 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.750+0000 7ff5b6035700 1 Processor -- start 2023-10-07T07:58:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.750+0000 7ff5b6035700 1 -- start start 2023-10-07T07:58:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5b6035700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5b006e2c0 0x7ff5b014f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5b6035700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 0x7ff5b00b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5b6035700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b01562f0 0x7ff5b00c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5b6035700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0164fe0 con 0x7ff5b0155f20 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5b6035700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0165880 con 0x7ff5b01562f0 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5b6035700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00b2a10 con 0x7ff5b006e2c0 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5aeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 0x7ff5b00b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5aeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 0x7ff5b00b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48226/0 (socket says 172.21.15.7:48226) 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5aeffd700 1 -- 172.21.15.7:0/3686294061 learned_addr learned my addr 172.21.15.7:0/3686294061 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5affff700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b01562f0 0x7ff5b00c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.751+0000 7ff5af7fe700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5b006e2c0 0x7ff5b014f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5aeffd700 1 -- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5b006e2c0 msgr2=0x7ff5b014f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5aeffd700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5b006e2c0 0x7ff5b014f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5aeffd700 1 -- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b01562f0 msgr2=0x7ff5b00c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5aeffd700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b01562f0 0x7ff5b00c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5aeffd700 1 -- 172.21.15.7:0/3686294061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a0009420 con 0x7ff5b0155f20 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5affff700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b01562f0 0x7ff5b00c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5aeffd700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 0x7ff5b00b1fc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400d890 tx=0x7ff5a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5977fe700 1 -- 172.21.15.7:0/3686294061 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40125f0 con 0x7ff5b0155f20 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5977fe700 1 -- 172.21.15.7:0/3686294061 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a4012750 con 0x7ff5b0155f20 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b0165230 con 0x7ff5b0155f20 2023-10-07T07:58:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5977fe700 1 -- 172.21.15.7:0/3686294061 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4012b10 con 0x7ff5b0155f20 2023-10-07T07:58:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.752+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b014f590 con 0x7ff5b0155f20 2023-10-07T07:58:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.753+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b0165a10 con 0x7ff5b0155f20 2023-10-07T07:58:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.754+0000 7ff5977fe700 1 -- 172.21.15.7:0/3686294061 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5a401e460 con 0x7ff5b0155f20 2023-10-07T07:58:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.754+0000 7ff5977fe700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff59c066f30 0x7ff59c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.754+0000 7ff5977fe700 1 -- 172.21.15.7:0/3686294061 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff5a4084ab0 con 0x7ff5b0155f20 2023-10-07T07:58:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.754+0000 7ff5af7fe700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff59c066f30 0x7ff59c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.755+0000 7ff5af7fe700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff59c066f30 0x7ff59c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00093f0 tx=0x7ff5a0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:08.759+0000 7ff5977fe700 1 -- 172.21.15.7:0/3686294061 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5a4051d20 con 0x7ff5b0155f20 2023-10-07T07:58:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.006+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5b0153c20 con 0x7ff5b0155f20 2023-10-07T07:58:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.006+0000 7ff5977fe700 1 -- 172.21.15.7:0/3686294061 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff5a4055340 con 0x7ff5b0155f20 2023-10-07T07:58:09.007 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T07:58:09.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff59c066f30 msgr2=0x7ff59c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff59c066f30 0x7ff59c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00093f0 tx=0x7ff5a0017000 comp rx=0 tx=0).stop 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 msgr2=0x7ff5b00b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 0x7ff5b00b1fc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400d890 tx=0x7ff5a4008040 comp rx=0 tx=0).stop 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 shutdown_connections 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff59c066f30 0x7ff59c0693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b01562f0 0x7ff5b00c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5b006e2c0 0x7ff5b014f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 --2- 172.21.15.7:0/3686294061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b0155f20 0x7ff5b00b1fc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 >> 172.21.15.7:0/3686294061 conn(0x7ff5b0068bd0 msgr2=0x7ff5b00c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 shutdown_connections 2023-10-07T07:58:09.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:09.008+0000 7ff5b6035700 1 -- 172.21.15.7:0/3686294061 wait complete. 2023-10-07T07:58:09.022 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T07:58:09.023 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T07:58:09.023 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-07T07:58:09.023 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-07T07:58:09.023 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-07T07:58:09.023 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T07:58:09.023 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T07:58:09.023 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T07:58:09.023 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-07T07:58:09.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.023 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-07T07:58:09.044 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2023-10-07T07:58:09.045 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42446sec preferred_lft 42446sec 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2023-10-07T07:58:09.046 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:58:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 62:07:64:37:78:fc brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T07:58:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f00c:5eff:fefc:98cd/64 scope link 2023-10-07T07:58:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.047 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:58:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 62:07:64:37:78:fc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T07:58:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6007:64ff:fe37:78fc/64 scope link 2023-10-07T07:58:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.048 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T07:58:09.117 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-07T07:58:09.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.118 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-07T07:58:09.184 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-07T07:58:09.184 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T07:58:09.184 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-07T07:58:09.184 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-07T07:58:09.184 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T07:58:09.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.185 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:09.185 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:09.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T07:58:09.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T07:58:09.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T07:58:09.185 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:09.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.310 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-07T07:58:09.332 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T07:58:09.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.333 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2023-10-07T07:58:09.396 INFO:teuthology.orchestra.run.smithi007.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-07T07:58:09.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.396 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:09.396 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:09.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T07:58:09.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-07T07:58:09.397 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:09.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-07T07:58:09.556 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:09.556 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:09.556 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:09.557 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.557 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:09.557 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.557 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:58:09.557 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:ef:be:00:f1:17 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-07T07:58:09.557 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-07T07:58:09.557 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.557 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c8ef:beff:fe00:f117/64 scope link 2023-10-07T07:58:09.557 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-07T07:58:09.619 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:ef:be:00:f1:17 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c8ef:beff:fe00:f117/64 scope link 2023-10-07T07:58:09.620 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:09.635 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-07T07:58:09.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.636 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:09.636 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:09.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-07T07:58:09.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-07T07:58:09.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-07T07:58:09.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-07T07:58:09.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-07T07:58:09.636 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:09.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:09.839 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:09.840 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:09.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.1 up 2023-10-07T07:58:09.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.1 master ceph-brx 2023-10-07T07:58:09.840 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:09.952 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-07T07:58:09.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:09.952 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-07T07:58:09.972 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-07T07:58:09.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:09.973 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-07T07:58:10.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T07:58:10.111 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:10.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:10.186 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:10.187 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:10.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:10.188 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T07:58:10.251 INFO:teuthology.orchestra.run.smithi007.stdout:47 2023-10-07T07:58:10.252 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-10-07T07:58:10.252 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-07T07:58:10.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T07:58:10.330 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:10.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:10.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi007.stderr:2023-10-07T07:58:10.394+0000 7fd3ff883540 -1 init, newargv = 0x55b9173c6e00 newargc=15 2023-10-07T07:58:10.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi007.stderr:ceph-fuse[129592]: starting ceph client 2023-10-07T07:58:10.401 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi007.stderr:ceph-fuse[129592]: starting fuse 2023-10-07T07:58:10.405 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:10.406 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:10.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:10.407 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T07:58:10.468 INFO:teuthology.orchestra.run.smithi007.stdout:47 2023-10-07T07:58:10.468 INFO:teuthology.orchestra.run.smithi007.stdout:48 2023-10-07T07:58:10.469 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-10-07T07:58:10.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:10.469 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-07T07:58:10.469 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-07T07:58:10.469 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T07:58:10.469 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-07T07:58:10.469 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:10.469 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-07T07:58:10.470 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T07:58:10.471 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T07:58:10.471 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T07:58:10.471 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-07T07:58:10.471 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-07T07:58:10.471 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-07T07:58:10.471 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T07:58:10.471 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:10.471 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.2")) 2023-10-07T07:58:10.471 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T07:58:10.610 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.2.129592.asok 2023-10-07T07:58:10.791 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.129592.asok 2023-10-07T07:58:10.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:10.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.129592.asok status 2023-10-07T07:58:10.966 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:58:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-07T07:58:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T07:58:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T07:58:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "2", 2023-10-07T07:58:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-07T07:58:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-07T07:58:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "129592", 2023-10-07T07:58:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-07T07:58:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-07T07:58:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-07T07:58:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4680, 2023-10-07T07:58:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-07T07:58:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-07T07:58:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-07T07:58:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4680 2023-10-07T07:58:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T07:58:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:58:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.2:0", 2023-10-07T07:58:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2388481625 2023-10-07T07:58:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T07:58:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:58:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.2:0", 2023-10-07T07:58:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2388481625 2023-10-07T07:58:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4680 192.168.0.2:0/2388481625", 2023-10-07T07:58:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.2:0/2388481625", 2023-10-07T07:58:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-07T07:58:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 8, 2023-10-07T07:58:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 21, 2023-10-07T07:58:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-07T07:58:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-07T07:58:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-07T07:58:10.973 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:58:10.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T07:58:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.110+0000 7fe05e07e700 1 Processor -- start 2023-10-07T07:58:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.110+0000 7fe05e07e700 1 -- start start 2023-10-07T07:58:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.110+0000 7fe05e07e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe0581592c0 0x7fe05806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.110+0000 7fe05e07e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 0x7fe05806a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.110+0000 7fe05e07e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05806bb70 con 0x7fe058159690 2023-10-07T07:58:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.110+0000 7fe05e07e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0580cda10 con 0x7fe0581592c0 2023-10-07T07:58:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.110+0000 7fe05e07e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05806af60 con 0x7fe0580cb290 2023-10-07T07:58:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe0581592c0 0x7fe05806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe0581592c0 0x7fe05806a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35878/0 (socket says 172.21.15.7:35878) 2023-10-07T07:58:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe056ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe058159690 0x7fe0580cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55198/0 (socket says 172.21.15.7:55198) 2023-10-07T07:58:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0577fe700 1 -- 172.21.15.7:0/1723525139 learned_addr learned my addr 172.21.15.7:0/1723525139 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe057fff700 1 --2- 172.21.15.7:0/1723525139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 0x7fe05806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0577fe700 1 -- 172.21.15.7:0/1723525139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 msgr2=0x7fe05806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0577fe700 1 --2- 172.21.15.7:0/1723525139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 0x7fe05806a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0577fe700 1 -- 172.21.15.7:0/1723525139 >> v1:172.21.15.7:6789/0 conn(0x7fe058159690 legacy=0x7fe0580cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0577fe700 1 -- 172.21.15.7:0/1723525139 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05806b240 con 0x7fe0581592c0 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0567fc700 1 -- 172.21.15.7:0/1723525139 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351272274 0 0) 0x7fe05806bb70 con 0x7fe058159690 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0577fe700 1 --2- 172.21.15.7:0/1723525139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 0x7fe05806a3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0480099b0 tx=0x7fe048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee8b71344f6b727 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0567fc700 1 -- 172.21.15.7:0/1723525139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800b860 con 0x7fe0581592c0 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.111+0000 7fe0567fc700 1 -- 172.21.15.7:0/1723525139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04800b9c0 con 0x7fe0581592c0 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.112+0000 7fe0567fc700 1 -- 172.21.15.7:0/1723525139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800bd60 con 0x7fe0581592c0 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.112+0000 7fe05e07e700 1 -- 172.21.15.7:0/1723525139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 msgr2=0x7fe05806a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.112+0000 7fe05e07e700 1 --2- 172.21.15.7:0/1723525139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 0x7fe05806a3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0480099b0 tx=0x7fe048009250 comp rx=0 tx=0).stop 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.112+0000 7fe05e07e700 1 -- 172.21.15.7:0/1723525139 shutdown_connections 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.112+0000 7fe05e07e700 1 --2- 172.21.15.7:0/1723525139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 0x7fe05806a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.112+0000 7fe05e07e700 1 --2- 172.21.15.7:0/1723525139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 0x7fe05806a3f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.112+0000 7fe05e07e700 1 -- 172.21.15.7:0/1723525139 >> 172.21.15.7:0/1723525139 conn(0x7fe058068bd0 msgr2=0x7fe0580ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.112+0000 7fe05e07e700 1 -- 172.21.15.7:0/1723525139 shutdown_connections 2023-10-07T07:58:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.112+0000 7fe05e07e700 1 -- 172.21.15.7:0/1723525139 wait complete. 2023-10-07T07:58:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.113+0000 7fe05e07e700 1 Processor -- start 2023-10-07T07:58:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.114+0000 7fe05e07e700 1 -- start start 2023-10-07T07:58:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.114+0000 7fe05e07e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 0x7fe05815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.114+0000 7fe05e07e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 0x7fe05815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.114+0000 7fe05e07e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe058159690 0x7fe05813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.114+0000 7fe05e07e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05815ae90 con 0x7fe0581592c0 2023-10-07T07:58:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.114+0000 7fe05e07e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05815ea70 con 0x7fe058159690 2023-10-07T07:58:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.114+0000 7fe05e07e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe058159fa0 con 0x7fe0580cb290 2023-10-07T07:58:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.114+0000 7fe056ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 0x7fe05815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe056ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 0x7fe05815c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35884/0 (socket says 172.21.15.7:35884) 2023-10-07T07:58:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe056ffd700 1 -- 172.21.15.7:0/2978258999 learned_addr learned my addr 172.21.15.7:0/2978258999 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe0577fe700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 0x7fe05815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe057fff700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe058159690 0x7fe05813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe056ffd700 1 -- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 msgr2=0x7fe05815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe056ffd700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 0x7fe05815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe056ffd700 1 -- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe058159690 msgr2=0x7fe05813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe056ffd700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe058159690 0x7fe05813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe056ffd700 1 -- 172.21.15.7:0/2978258999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0480094f0 con 0x7fe0581592c0 2023-10-07T07:58:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe056ffd700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 0x7fe05815c790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c7e0 tx=0x7fe040007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe04f7fe700 1 -- 172.21.15.7:0/2978258999 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040005a70 con 0x7fe0581592c0 2023-10-07T07:58:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe04f7fe700 1 -- 172.21.15.7:0/2978258999 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe040005bd0 con 0x7fe0581592c0 2023-10-07T07:58:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05815a280 con 0x7fe0581592c0 2023-10-07T07:58:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe04f7fe700 1 -- 172.21.15.7:0/2978258999 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0400070d0 con 0x7fe0581592c0 2023-10-07T07:58:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.115+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05815a8c0 con 0x7fe0581592c0 2023-10-07T07:58:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.116+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05815b040 con 0x7fe0581592c0 2023-10-07T07:58:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.117+0000 7fe04f7fe700 1 -- 172.21.15.7:0/2978258999 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe040007440 con 0x7fe0581592c0 2023-10-07T07:58:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.117+0000 7fe04f7fe700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe038066f30 0x7fe0380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.117+0000 7fe04f7fe700 1 -- 172.21.15.7:0/2978258999 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe040083a30 con 0x7fe0581592c0 2023-10-07T07:58:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.117+0000 7fe0577fe700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe038066f30 0x7fe0380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.118+0000 7fe0577fe700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe038066f30 0x7fe0380693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0480099b0 tx=0x7fe048017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.122+0000 7fe04f7fe700 1 -- 172.21.15.7:0/2978258999 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe05815b040 con 0x7fe0581592c0 2023-10-07T07:58:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.367+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe05815b040 con 0x7fe0581592c0 2023-10-07T07:58:11.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.367+0000 7fe04f7fe700 1 -- 172.21.15.7:0/2978258999 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe05815b040 con 0x7fe0581592c0 2023-10-07T07:58:11.369 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe038066f30 msgr2=0x7fe0380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe038066f30 0x7fe0380693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0480099b0 tx=0x7fe048017000 comp rx=0 tx=0).stop 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 msgr2=0x7fe05815c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 0x7fe05815c790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c7e0 tx=0x7fe040007e10 comp rx=0 tx=0).stop 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 shutdown_connections 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe038066f30 0x7fe0380693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe058159690 0x7fe05813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0580cb290 0x7fe05815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 --2- 172.21.15.7:0/2978258999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0581592c0 0x7fe05815c790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.369+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 >> 172.21.15.7:0/2978258999 conn(0x7fe058068bd0 msgr2=0x7fe0580673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.370+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 shutdown_connections 2023-10-07T07:58:11.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:11.370+0000 7fe05e07e700 1 -- 172.21.15.7:0/2978258999 wait complete. 2023-10-07T07:58:11.384 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T07:58:11.384 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T07:58:11.384 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-10-07T07:58:11.384 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-07T07:58:11.384 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:11.384 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T07:58:11.384 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T07:58:11.384 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T07:58:11.385 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T07:58:11.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:11.385 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-10-07T07:58:11.408 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:11.408 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:11.408 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:11.408 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:11.408 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:11.408 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:11.408 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:11.408 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:11.408 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42431sec preferred_lft 42431sec 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2023-10-07T07:58:11.409 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:11.410 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-07T07:58:11.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:11.410 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:11.410 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:11.410 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add name ceph-brx type bridge 2023-10-07T07:58:11.410 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr flush dev ceph-brx 2023-10-07T07:58:11.410 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set ceph-brx up 2023-10-07T07:58:11.410 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-07T07:58:11.410 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:11.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:11.578 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T07:58:11.614 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-10-07T07:58:11.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:11.616 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-10-07T07:58:11.694 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-10-07T07:58:11.694 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T07:58:11.695 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T07:58:11.695 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-07T07:58:11.695 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T07:58:11.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:11.695 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:11.695 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:11.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T07:58:11.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T07:58:11.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T07:58:11.696 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:12.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:12.015 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-10-07T07:58:12.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:12.033 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2023-10-07T07:58:12.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:12.091 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:12.092 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:12.092 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T07:58:12.092 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-07T07:58:12.092 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:12.187 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-07T07:58:12.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:12.188 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:12.188 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:12.188 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-07T07:58:12.188 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-07T07:58:12.188 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-07T07:58:12.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-07T07:58:12.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-07T07:58:12.189 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:12.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:12.410 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:12.410 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:12.410 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2023-10-07T07:58:12.410 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.0 master ceph-brx 2023-10-07T07:58:12.410 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:12.508 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-07T07:58:12.508 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:12.508 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:12.526 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-07T07:58:12.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:12.526 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:12.584 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2023-10-07T07:58:12.652 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:12.652 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:12.720 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:12.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:12.721 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2023-10-07T07:58:12.779 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T07:58:12.779 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-07T07:58:12.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2023-10-07T07:58:12.847 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:12.848 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:12.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:2023-10-07T07:58:12.904+0000 7f085490a540 -1 init, newargv = 0x562f8f3247c0 newargc=15 2023-10-07T07:58:12.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:ceph-fuse[126823]: starting ceph client 2023-10-07T07:58:12.910 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:ceph-fuse[126823]: starting fuse 2023-10-07T07:58:12.916 INFO:teuthology.orchestra.run.smithi172.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:12.917 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:12.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:12.918 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2023-10-07T07:58:12.977 INFO:teuthology.orchestra.run.smithi172.stdout:46 2023-10-07T07:58:12.977 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-07T07:58:12.977 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:12.977 DEBUG:teuthology.orchestra.run.smithi172:> import glob 2023-10-07T07:58:12.978 DEBUG:teuthology.orchestra.run.smithi172:> import re 2023-10-07T07:58:12.978 DEBUG:teuthology.orchestra.run.smithi172:> import os 2023-10-07T07:58:12.978 DEBUG:teuthology.orchestra.run.smithi172:> import subprocess 2023-10-07T07:58:12.978 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:12.978 DEBUG:teuthology.orchestra.run.smithi172:> def _find_admin_socket(client_name): 2023-10-07T07:58:12.978 DEBUG:teuthology.orchestra.run.smithi172:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-07T07:58:12.979 DEBUG:teuthology.orchestra.run.smithi172:> files = glob.glob(asok_path) 2023-10-07T07:58:12.979 DEBUG:teuthology.orchestra.run.smithi172:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-07T07:58:12.979 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:12.979 DEBUG:teuthology.orchestra.run.smithi172:> # Given a non-glob path, it better be there 2023-10-07T07:58:12.979 DEBUG:teuthology.orchestra.run.smithi172:> if "*" not in asok_path: 2023-10-07T07:58:12.979 DEBUG:teuthology.orchestra.run.smithi172:> assert(len(files) == 1) 2023-10-07T07:58:12.979 DEBUG:teuthology.orchestra.run.smithi172:> return files[0] 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> for f in files: 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> contents = proc_f.read() 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> if mountpoint in contents: 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> return f 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:12.980 DEBUG:teuthology.orchestra.run.smithi172:> print(_find_admin_socket("client.1")) 2023-10-07T07:58:12.981 DEBUG:teuthology.orchestra.run.smithi172:> ' 2023-10-07T07:58:13.100 INFO:teuthology.orchestra.run.smithi172.stdout:/var/run/ceph/ceph-client.1.126823.asok 2023-10-07T07:58:13.283 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.126823.asok 2023-10-07T07:58:13.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:13.283 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.126823.asok status 2023-10-07T07:58:13.437 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2023-10-07T07:58:13.437 INFO:teuthology.orchestra.run.smithi172.stdout: "metadata": { 2023-10-07T07:58:13.437 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T07:58:13.438 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T07:58:13.438 INFO:teuthology.orchestra.run.smithi172.stdout: "entity_id": "1", 2023-10-07T07:58:13.438 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": "smithi172", 2023-10-07T07:58:13.438 INFO:teuthology.orchestra.run.smithi172.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-07T07:58:13.438 INFO:teuthology.orchestra.run.smithi172.stdout: "pid": "126823", 2023-10-07T07:58:13.438 INFO:teuthology.orchestra.run.smithi172.stdout: "root": "/" 2023-10-07T07:58:13.438 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:13.438 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_count": 0, 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_pinned_count": 0, 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 4630, 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "inst": { 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "name": { 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "client", 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "num": 4630 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:58:13.439 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 3088706365 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: } 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 3088706365 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: "inst_str": "client.4630 192.168.0.1:0/3088706365", 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: "addr_str": "192.168.0.1:0/3088706365", 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: "inode_count": 1, 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_epoch": 8, 2023-10-07T07:58:13.440 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch": 21, 2023-10-07T07:58:13.441 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch_barrier": 0, 2023-10-07T07:58:13.441 INFO:teuthology.orchestra.run.smithi172.stdout: "blocklisted": false, 2023-10-07T07:58:13.441 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs" 2023-10-07T07:58:13.441 INFO:teuthology.orchestra.run.smithi172.stdout:} 2023-10-07T07:58:13.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T07:58:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.586+0000 7fdbd2874700 1 Processor -- start 2023-10-07T07:58:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.586+0000 7fdbd2874700 1 -- start start 2023-10-07T07:58:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.587+0000 7fdbd2874700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdbcc0aaa50 0x7fdbcc164110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.587+0000 7fdbd2874700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbcc06bb70 con 0x7fdbcc159690 2023-10-07T07:58:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.587+0000 7fdbd2874700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbcc0cda10 con 0x7fdbcc1592c0 2023-10-07T07:58:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.587+0000 7fdbd2874700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbcc06af60 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.587+0000 7fdbcb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdbcc159690 0x7fdbcc0cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55206/0 (socket says 172.21.15.7:55206) 2023-10-07T07:58:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.587+0000 7fdbcb7fe700 1 -- 172.21.15.7:0/137386493 learned_addr learned my addr 172.21.15.7:0/137386493 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.587+0000 7fdbd0e11700 1 --2- 172.21.15.7:0/137386493 >> v2:172.21.15.172:3300/0 conn(0x7fdbcc0aaa50 0x7fdbcc164110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.587+0000 7fdbcaffd700 1 -- 172.21.15.7:0/137386493 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038201892 0 0) 0x7fdbcc06bb70 con 0x7fdbcc159690 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.587+0000 7fdbcaffd700 1 -- 172.21.15.7:0/137386493 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb8003570 con 0x7fdbcc159690 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.588+0000 7fdbcaffd700 1 -- 172.21.15.7:0/137386493 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772954583 0 0) 0x7fdbcc0cda10 con 0x7fdbcc1592c0 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.588+0000 7fdbcaffd700 1 -- 172.21.15.7:0/137386493 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbcc06bb70 con 0x7fdbcc1592c0 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.588+0000 7fdbd0e11700 1 -- 172.21.15.7:0/137386493 >> v1:172.21.15.172:6789/0 conn(0x7fdbcc1592c0 legacy=0x7fdbcc06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.588+0000 7fdbd0e11700 1 -- 172.21.15.7:0/137386493 >> v1:172.21.15.7:6789/0 conn(0x7fdbcc159690 legacy=0x7fdbcc0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.588+0000 7fdbd0e11700 1 -- 172.21.15.7:0/137386493 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc06b240 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.588+0000 7fdbd0e11700 1 --2- 172.21.15.7:0/137386493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbcc0aaa50 0x7fdbcc164110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009e60 tx=0x7fdbc4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503c758aa1a1326f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.588+0000 7fdbcaffd700 1 -- 172.21.15.7:0/137386493 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400e070 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.588+0000 7fdbcaffd700 1 -- 172.21.15.7:0/137386493 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc400bab0 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.588+0000 7fdbcaffd700 1 -- 172.21.15.7:0/137386493 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400be70 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.589+0000 7fdbd2874700 1 -- 172.21.15.7:0/137386493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbcc0aaa50 msgr2=0x7fdbcc164110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:13.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.589+0000 7fdbd2874700 1 --2- 172.21.15.7:0/137386493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbcc0aaa50 0x7fdbcc164110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009e60 tx=0x7fdbc4009780 comp rx=0 tx=0).stop 2023-10-07T07:58:13.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.589+0000 7fdbd2874700 1 -- 172.21.15.7:0/137386493 shutdown_connections 2023-10-07T07:58:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.589+0000 7fdbd2874700 1 --2- 172.21.15.7:0/137386493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbcc0aaa50 0x7fdbcc164110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.589+0000 7fdbd2874700 1 -- 172.21.15.7:0/137386493 >> 172.21.15.7:0/137386493 conn(0x7fdbcc068bd0 msgr2=0x7fdbcc0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.589+0000 7fdbd2874700 1 -- 172.21.15.7:0/137386493 shutdown_connections 2023-10-07T07:58:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.589+0000 7fdbd2874700 1 -- 172.21.15.7:0/137386493 wait complete. 2023-10-07T07:58:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.590+0000 7fdbd2874700 1 Processor -- start 2023-10-07T07:58:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.590+0000 7fdbd2874700 1 -- start start 2023-10-07T07:58:13.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbd2874700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbcc0aaa50 0x7fdbcc0a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:13.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbd2874700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbcc1592c0 0x7fdbcc0a8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:13.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbd2874700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbcc159690 0x7fdbcc16d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:13.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbd2874700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc16e790 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbd2874700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc0a5480 con 0x7fdbcc159690 2023-10-07T07:58:13.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbd2874700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc16ea30 con 0x7fdbcc1592c0 2023-10-07T07:58:13.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbcbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbcc0aaa50 0x7fdbcc0a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:13.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbcbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbcc0aaa50 0x7fdbcc0a7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35898/0 (socket says 172.21.15.7:35898) 2023-10-07T07:58:13.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbcbfff700 1 -- 172.21.15.7:0/3190390804 learned_addr learned my addr 172.21.15.7:0/3190390804 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbcb7fe700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbcc1592c0 0x7fdbcc0a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbd0e11700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbcc159690 0x7fdbcc16d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbcbfff700 1 -- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbcc1592c0 msgr2=0x7fdbcc0a8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbcbfff700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbcc1592c0 0x7fdbcc0a8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbcbfff700 1 -- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbcc159690 msgr2=0x7fdbcc16d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbcbfff700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbcc159690 0x7fdbcc16d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.591+0000 7fdbcbfff700 1 -- 172.21.15.7:0/3190390804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc4009a20 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.592+0000 7fdbcbfff700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbcc0aaa50 0x7fdbcc0a7bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c790 tx=0x7fdbbc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.592+0000 7fdbc3fff700 1 -- 172.21.15.7:0/3190390804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc010510 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.592+0000 7fdbc3fff700 1 -- 172.21.15.7:0/3190390804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc010670 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.592+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbcc16ed10 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.592+0000 7fdbc3fff700 1 -- 172.21.15.7:0/3190390804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc010a10 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.592+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbcc16f2d0 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.593+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbcc0716c0 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.593+0000 7fdbc3fff700 1 -- 172.21.15.7:0/3190390804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdbbc01b3b0 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.594+0000 7fdbc3fff700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdbac066f30 0x7fdbac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.594+0000 7fdbc3fff700 1 -- 172.21.15.7:0/3190390804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdbbc083dd0 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.594+0000 7fdbcb7fe700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdbac066f30 0x7fdbac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.594+0000 7fdbcb7fe700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdbac066f30 0x7fdbac0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0aecb0 tx=0x7fdbb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.598+0000 7fdbc3fff700 1 -- 172.21.15.7:0/3190390804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbbc051040 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.850+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdbcc0c2560 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.850+0000 7fdbc3fff700 1 -- 172.21.15.7:0/3190390804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdbbc085070 con 0x7fdbcc0aaa50 2023-10-07T07:58:13.852 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T07:58:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdbac066f30 msgr2=0x7fdbac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdbac066f30 0x7fdbac0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0aecb0 tx=0x7fdbb4009040 comp rx=0 tx=0).stop 2023-10-07T07:58:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbcc0aaa50 msgr2=0x7fdbcc0a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbcc0aaa50 0x7fdbcc0a7bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c790 tx=0x7fdbbc007e10 comp rx=0 tx=0).stop 2023-10-07T07:58:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 shutdown_connections 2023-10-07T07:58:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdbac066f30 0x7fdbac0693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbcc159690 0x7fdbcc16d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbcc0aaa50 0x7fdbcc0a7bf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 --2- 172.21.15.7:0/3190390804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbcc1592c0 0x7fdbcc0a8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 >> 172.21.15.7:0/3190390804 conn(0x7fdbcc068bd0 msgr2=0x7fdbcc0c32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 shutdown_connections 2023-10-07T07:58:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:13.853+0000 7fdbd2874700 1 -- 172.21.15.7:0/3190390804 wait complete. 2023-10-07T07:58:13.867 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T07:58:13.867 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T07:58:13.867 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-10-07T07:58:13.867 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-07T07:58:13.867 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-07T07:58:13.868 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T07:58:13.868 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T07:58:13.868 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T07:58:13.868 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-07T07:58:13.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:13.868 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2023-10-07T07:58:13.888 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42429sec preferred_lft 42429sec 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:58:13.889 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 36:e1:63:77:36:7f brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:13.890 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T07:58:13.890 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:13.890 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2:8eff:fee3:e22b/64 scope link 2023-10-07T07:58:13.890 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:13.890 INFO:teuthology.orchestra.run.smithi172.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:58:13.890 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 36:e1:63:77:36:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T07:58:13.890 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::34e1:63ff:fe77:367f/64 scope link tentative 2023-10-07T07:58:13.890 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:13.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:13.890 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T07:58:13.955 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-10-07T07:58:13.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:13.956 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-10-07T07:58:14.016 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-10-07T07:58:14.016 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T07:58:14.016 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T07:58:14.016 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-07T07:58:14.016 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T07:58:14.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:14.017 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:14.017 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:14.017 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T07:58:14.017 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T07:58:14.017 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T07:58:14.017 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:14.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:14.122 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-10-07T07:58:14.140 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-07T07:58:14.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:14.141 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2023-10-07T07:58:14.199 INFO:teuthology.orchestra.run.smithi172.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-07T07:58:14.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:14.199 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:14.200 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:14.200 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-07T07:58:14.200 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-07T07:58:14.200 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:14.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:14.293 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-07T07:58:14.335 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:14.335 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:14.335 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:14.335 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:14.335 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:14.336 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:14.336 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:58:14.336 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0a:b6:9f:2a:99:8d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-07T07:58:14.336 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-07T07:58:14.336 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:14.336 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::8b6:9fff:fe2a:998d/64 scope link 2023-10-07T07:58:14.336 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:14.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:14.343 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-07T07:58:14.425 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:14.425 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:14.425 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:14.425 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:14.425 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:14.425 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:14.426 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:58:14.426 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0a:b6:9f:2a:99:8d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-07T07:58:14.426 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-07T07:58:14.426 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:14.426 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::8b6:9fff:fe2a:998d/64 scope link 2023-10-07T07:58:14.426 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:14.437 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-07T07:58:14.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:14.437 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:14.438 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:14.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-07T07:58:14.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-07T07:58:14.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-07T07:58:14.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-07T07:58:14.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-07T07:58:14.438 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:14.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:14.669 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:14.669 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:14.669 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.1 up 2023-10-07T07:58:14.669 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.1 master ceph-brx 2023-10-07T07:58:14.669 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:14.767 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-07T07:58:14.767 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:14.767 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-07T07:58:14.785 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-07T07:58:14.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:14.785 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-07T07:58:14.843 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2023-10-07T07:58:14.911 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:14.912 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:14.978 INFO:teuthology.orchestra.run.smithi172.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:14.979 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:14.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:14.980 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2023-10-07T07:58:15.038 INFO:teuthology.orchestra.run.smithi172.stdout:46 2023-10-07T07:58:15.039 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-10-07T07:58:15.039 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-07T07:58:15.081 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2023-10-07T07:58:15.108 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:15.108 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:15.165 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi172.stderr:2023-10-07T07:58:15.163+0000 7f1762540540 -1 init, newargv = 0x560d5648f240 newargc=15 2023-10-07T07:58:15.165 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi172.stderr:ceph-fuse[127498]: starting ceph client 2023-10-07T07:58:15.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi172.stderr:ceph-fuse[127498]: starting fuse 2023-10-07T07:58:15.178 INFO:teuthology.orchestra.run.smithi172.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:15.179 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:15.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:15.179 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2023-10-07T07:58:15.238 INFO:teuthology.orchestra.run.smithi172.stdout:46 2023-10-07T07:58:15.239 INFO:teuthology.orchestra.run.smithi172.stdout:48 2023-10-07T07:58:15.239 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-10-07T07:58:15.239 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:15.239 DEBUG:teuthology.orchestra.run.smithi172:> import glob 2023-10-07T07:58:15.239 DEBUG:teuthology.orchestra.run.smithi172:> import re 2023-10-07T07:58:15.239 DEBUG:teuthology.orchestra.run.smithi172:> import os 2023-10-07T07:58:15.239 DEBUG:teuthology.orchestra.run.smithi172:> import subprocess 2023-10-07T07:58:15.239 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> def _find_admin_socket(client_name): 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> files = glob.glob(asok_path) 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> # Given a non-glob path, it better be there 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> if "*" not in asok_path: 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> assert(len(files) == 1) 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> return files[0] 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> for f in files: 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T07:58:15.240 DEBUG:teuthology.orchestra.run.smithi172:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T07:58:15.241 DEBUG:teuthology.orchestra.run.smithi172:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T07:58:15.241 DEBUG:teuthology.orchestra.run.smithi172:> contents = proc_f.read() 2023-10-07T07:58:15.241 DEBUG:teuthology.orchestra.run.smithi172:> if mountpoint in contents: 2023-10-07T07:58:15.241 DEBUG:teuthology.orchestra.run.smithi172:> return f 2023-10-07T07:58:15.241 DEBUG:teuthology.orchestra.run.smithi172:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T07:58:15.241 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:15.241 DEBUG:teuthology.orchestra.run.smithi172:> print(_find_admin_socket("client.3")) 2023-10-07T07:58:15.241 DEBUG:teuthology.orchestra.run.smithi172:> ' 2023-10-07T07:58:15.364 INFO:teuthology.orchestra.run.smithi172.stdout:/var/run/ceph/ceph-client.3.127498.asok 2023-10-07T07:58:15.546 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.127498.asok 2023-10-07T07:58:15.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:15.546 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.127498.asok status 2023-10-07T07:58:15.700 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2023-10-07T07:58:15.700 INFO:teuthology.orchestra.run.smithi172.stdout: "metadata": { 2023-10-07T07:58:15.700 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T07:58:15.700 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T07:58:15.700 INFO:teuthology.orchestra.run.smithi172.stdout: "entity_id": "3", 2023-10-07T07:58:15.700 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": "smithi172", 2023-10-07T07:58:15.700 INFO:teuthology.orchestra.run.smithi172.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-07T07:58:15.700 INFO:teuthology.orchestra.run.smithi172.stdout: "pid": "127498", 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: "root": "/" 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_count": 0, 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_pinned_count": 0, 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 4499, 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: "inst": { 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: "name": { 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "client", 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: "num": 4499 2023-10-07T07:58:15.701 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:15.702 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2023-10-07T07:58:15.702 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2023-10-07T07:58:15.702 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.2:0", 2023-10-07T07:58:15.702 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 509826547 2023-10-07T07:58:15.702 INFO:teuthology.orchestra.run.smithi172.stdout: } 2023-10-07T07:58:15.702 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:15.702 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2023-10-07T07:58:15.702 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2023-10-07T07:58:15.702 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.2:0", 2023-10-07T07:58:15.703 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 509826547 2023-10-07T07:58:15.703 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:15.703 INFO:teuthology.orchestra.run.smithi172.stdout: "inst_str": "client.4499 192.168.0.2:0/509826547", 2023-10-07T07:58:15.703 INFO:teuthology.orchestra.run.smithi172.stdout: "addr_str": "192.168.0.2:0/509826547", 2023-10-07T07:58:15.703 INFO:teuthology.orchestra.run.smithi172.stdout: "inode_count": 1, 2023-10-07T07:58:15.703 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_epoch": 8, 2023-10-07T07:58:15.703 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch": 21, 2023-10-07T07:58:15.704 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch_barrier": 0, 2023-10-07T07:58:15.704 INFO:teuthology.orchestra.run.smithi172.stdout: "blocklisted": false, 2023-10-07T07:58:15.704 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs" 2023-10-07T07:58:15.704 INFO:teuthology.orchestra.run.smithi172.stdout:} 2023-10-07T07:58:15.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:15.710 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-07T07:58:15.710 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-07T07:58:15.735 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:15.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:15.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:15.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:15.812 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-07T07:58:15.812 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-07T07:58:15.876 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-07T07:58:15.876 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-07T07:58:15.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:15.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-07T07:58:15.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:15.956 DEBUG:teuthology.orchestra.run.smithi172:> stat --file-system '--printf=%T 2023-10-07T07:58:15.956 DEBUG:teuthology.orchestra.run.smithi172:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:15.974 INFO:teuthology.orchestra.run.smithi172.stdout:fuseblk 2023-10-07T07:58:15.975 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:15.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:15.975 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:16.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:16.043 DEBUG:teuthology.orchestra.run.smithi172:> stat --file-system '--printf=%T 2023-10-07T07:58:16.043 DEBUG:teuthology.orchestra.run.smithi172:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-07T07:58:16.101 INFO:teuthology.orchestra.run.smithi172.stdout:fuseblk 2023-10-07T07:58:16.101 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-07T07:58:16.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:16.102 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-07T07:58:16.169 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-07T07:58:16.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.329+0000 7fb61b0a5700 1 Processor -- start 2023-10-07T07:58:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.329+0000 7fb61b0a5700 1 -- start start 2023-10-07T07:58:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb61b0a5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6141592c0 0x7fb61406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb61b0a5700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb614159690 0x7fb61406a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb61b0a5700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61406bb70 con 0x7fb6140cab80 2023-10-07T07:58:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb61b0a5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6140cda10 con 0x7fb6141592c0 2023-10-07T07:58:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb61b0a5700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61406af60 con 0x7fb614159690 2023-10-07T07:58:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb618e41700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6141592c0 0x7fb61406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb618e41700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6141592c0 0x7fb61406a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35906/0 (socket says 172.21.15.7:35906) 2023-10-07T07:58:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb618e41700 1 -- 172.21.15.7:0/3212732841 learned_addr learned my addr 172.21.15.7:0/3212732841 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb613fff700 1 --2- 172.21.15.7:0/3212732841 >> v2:172.21.15.172:3300/0 conn(0x7fb614159690 0x7fb61406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb618e41700 1 -- 172.21.15.7:0/3212732841 >> v2:172.21.15.172:3300/0 conn(0x7fb614159690 msgr2=0x7fb61406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb618e41700 1 --2- 172.21.15.7:0/3212732841 >> v2:172.21.15.172:3300/0 conn(0x7fb614159690 0x7fb61406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb618e41700 1 -- 172.21.15.7:0/3212732841 >> v1:172.21.15.172:6789/0 conn(0x7fb6140cab80 legacy=0x7fb6140caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.330+0000 7fb618e41700 1 -- 172.21.15.7:0/3212732841 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61406b240 con 0x7fb6141592c0 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb618e41700 1 --2- 172.21.15.7:0/3212732841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6141592c0 0x7fb61406a3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb6140caa90 tx=0x7fb608009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e679a62bdf36099e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb6137fe700 1 -- 172.21.15.7:0/3212732841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608009e70 con 0x7fb6141592c0 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb6137fe700 1 -- 172.21.15.7:0/3212732841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60800b860 con 0x7fb6141592c0 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb6137fe700 1 -- 172.21.15.7:0/3212732841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60800bc00 con 0x7fb6141592c0 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb61b0a5700 1 -- 172.21.15.7:0/3212732841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6141592c0 msgr2=0x7fb61406a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb61b0a5700 1 --2- 172.21.15.7:0/3212732841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6141592c0 0x7fb61406a3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb6140caa90 tx=0x7fb608009250 comp rx=0 tx=0).stop 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb61b0a5700 1 -- 172.21.15.7:0/3212732841 shutdown_connections 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb61b0a5700 1 --2- 172.21.15.7:0/3212732841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6141592c0 0x7fb61406a3f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb61b0a5700 1 --2- 172.21.15.7:0/3212732841 >> v2:172.21.15.172:3300/0 conn(0x7fb614159690 0x7fb61406a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb61b0a5700 1 -- 172.21.15.7:0/3212732841 >> 172.21.15.7:0/3212732841 conn(0x7fb614068bd0 msgr2=0x7fb6140ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb61b0a5700 1 -- 172.21.15.7:0/3212732841 shutdown_connections 2023-10-07T07:58:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.331+0000 7fb61b0a5700 1 -- 172.21.15.7:0/3212732841 wait complete. 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.333+0000 7fb61b0a5700 1 Processor -- start 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.333+0000 7fb61b0a5700 1 -- start start 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb61b0a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6140cab80 0x7fb6140ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb61b0a5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6141592c0 0x7fb614150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb61b0a5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb614159690 0x7fb61416a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb61b0a5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb614151520 con 0x7fb6140cab80 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb61b0a5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140addb0 con 0x7fb614159690 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb61b0a5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61416ac00 con 0x7fb6141592c0 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb618e41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6140cab80 0x7fb6140ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb618e41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6140cab80 0x7fb6140ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35922/0 (socket says 172.21.15.7:35922) 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb618e41700 1 -- 172.21.15.7:0/2493512427 learned_addr learned my addr 172.21.15.7:0/2493512427 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb619642700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb614159690 0x7fb61416a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb613fff700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6141592c0 0x7fb614150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb618e41700 1 -- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6141592c0 msgr2=0x7fb614150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb618e41700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6141592c0 0x7fb614150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb618e41700 1 -- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb614159690 msgr2=0x7fb61416a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb618e41700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb614159690 0x7fb61416a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb618e41700 1 -- 172.21.15.7:0/2493512427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6080094f0 con 0x7fb6140cab80 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.334+0000 7fb618e41700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6140cab80 0x7fb6140ae240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb614069020 tx=0x7fb608017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.335+0000 7fb610ff9700 1 -- 172.21.15.7:0/2493512427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60800bbe0 con 0x7fb6140cab80 2023-10-07T07:58:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.335+0000 7fb610ff9700 1 -- 172.21.15.7:0/2493512427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608011420 con 0x7fb6140cab80 2023-10-07T07:58:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.335+0000 7fb610ff9700 1 -- 172.21.15.7:0/2493512427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6080117c0 con 0x7fb6140cab80 2023-10-07T07:58:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.335+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6140ad1a0 con 0x7fb6140cab80 2023-10-07T07:58:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.335+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6140ad760 con 0x7fb6140cab80 2023-10-07T07:58:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.336+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6140ae780 con 0x7fb6140cab80 2023-10-07T07:58:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.336+0000 7fb610ff9700 1 -- 172.21.15.7:0/2493512427 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb608011bf0 con 0x7fb6140cab80 2023-10-07T07:58:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.336+0000 7fb610ff9700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb5f8066f30 0x7fb5f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.337+0000 7fb610ff9700 1 -- 172.21.15.7:0/2493512427 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb608085970 con 0x7fb6140cab80 2023-10-07T07:58:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.337+0000 7fb613fff700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb5f8066f30 0x7fb5f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.337+0000 7fb613fff700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb5f8066f30 0x7fb5f80693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb600009db0 tx=0x7fb6000091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.341+0000 7fb610ff9700 1 -- 172.21.15.7:0/2493512427 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb608052be0 con 0x7fb6140cab80 2023-10-07T07:58:16.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.541+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6140716c0 con 0x7fb6140cab80 2023-10-07T07:58:16.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.542+0000 7fb610ff9700 1 -- 172.21.15.7:0/2493512427 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fb608056200 con 0x7fb6140cab80 2023-10-07T07:58:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:16.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb5f8066f30 msgr2=0x7fb5f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb5f8066f30 0x7fb5f80693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb600009db0 tx=0x7fb6000091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6140cab80 msgr2=0x7fb6140ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6140cab80 0x7fb6140ae240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb614069020 tx=0x7fb608017000 comp rx=0 tx=0).stop 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 shutdown_connections 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb5f8066f30 0x7fb5f80693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb614159690 0x7fb61416a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6140cab80 0x7fb6140ae240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 --2- 172.21.15.7:0/2493512427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6141592c0 0x7fb614150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 >> 172.21.15.7:0/2493512427 conn(0x7fb614068bd0 msgr2=0x7fb614158930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 shutdown_connections 2023-10-07T07:58:16.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.544+0000 7fb61b0a5700 1 -- 172.21.15.7:0/2493512427 wait complete. 2023-10-07T07:58:16.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T07:57:48.285077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T07:57:49.111979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T07:58:16.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:58:16.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.694+0000 7f5bddb46700 1 Processor -- start 2023-10-07T07:58:16.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.694+0000 7f5bddb46700 1 -- start start 2023-10-07T07:58:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bddb46700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5bd813f840 0x7f5bd806e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bddb46700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 0x7f5bd80c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bddb46700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd80c7300 con 0x7f5bd8068d30 2023-10-07T07:58:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bddb46700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd806e580 con 0x7f5bd813f840 2023-10-07T07:58:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bddb46700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd806e840 con 0x7f5bd813fc10 2023-10-07T07:58:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bd77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5bd813f840 0x7f5bd806e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bd7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5bd8068d30 0x7f5bd8159c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55208/0 (socket says 172.21.15.7:55208) 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bd7fff700 1 -- 172.21.15.7:0/4012702120 learned_addr learned my addr 172.21.15.7:0/4012702120 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bd77fe700 1 --2- 172.21.15.7:0/4012702120 >> v2:172.21.15.7:3300/0 conn(0x7f5bd813f840 0x7f5bd806e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35932/0 (socket says 172.21.15.7:35932) 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.695+0000 7f5bd6ffd700 1 --2- 172.21.15.7:0/4012702120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 0x7f5bd80c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bd77fe700 1 -- 172.21.15.7:0/4012702120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 msgr2=0x7f5bd80c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bd77fe700 1 --2- 172.21.15.7:0/4012702120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 0x7f5bd80c64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bd77fe700 1 -- 172.21.15.7:0/4012702120 >> v1:172.21.15.7:6789/0 conn(0x7f5bd8068d30 legacy=0x7f5bd8159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bd77fe700 1 -- 172.21.15.7:0/4012702120 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd806eb20 con 0x7f5bd813f840 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bd77fe700 1 --2- 172.21.15.7:0/4012702120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd813f840 0x7f5bd806e040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800bb70 tx=0x7f5bc800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a0f62748807881 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bd67fc700 1 -- 172.21.15.7:0/4012702120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8005c10 con 0x7f5bd813f840 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bd67fc700 1 -- 172.21.15.7:0/4012702120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc8005d70 con 0x7f5bd813f840 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bd67fc700 1 -- 172.21.15.7:0/4012702120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc800da60 con 0x7f5bd813f840 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bddb46700 1 -- 172.21.15.7:0/4012702120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd813f840 msgr2=0x7f5bd806e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.696+0000 7f5bddb46700 1 --2- 172.21.15.7:0/4012702120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd813f840 0x7f5bd806e040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800bb70 tx=0x7f5bc800b3e0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.697+0000 7f5bddb46700 1 -- 172.21.15.7:0/4012702120 shutdown_connections 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.697+0000 7f5bddb46700 1 --2- 172.21.15.7:0/4012702120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd813f840 0x7f5bd806e040 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.697+0000 7f5bddb46700 1 --2- 172.21.15.7:0/4012702120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 0x7f5bd80c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.697+0000 7f5bddb46700 1 -- 172.21.15.7:0/4012702120 >> 172.21.15.7:0/4012702120 conn(0x7f5bd80cbe70 msgr2=0x7f5bd80cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.697+0000 7f5bddb46700 1 -- 172.21.15.7:0/4012702120 shutdown_connections 2023-10-07T07:58:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.697+0000 7f5bddb46700 1 -- 172.21.15.7:0/4012702120 wait complete. 2023-10-07T07:58:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.698+0000 7f5bddb46700 1 Processor -- start 2023-10-07T07:58:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.698+0000 7f5bddb46700 1 -- start start 2023-10-07T07:58:16.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bddb46700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8068d30 0x7f5bd8155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bddb46700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd813f840 0x7f5bd80abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bddb46700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 0x7f5bd80ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bddb46700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd80a9bf0 con 0x7f5bd8068d30 2023-10-07T07:58:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bddb46700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd80a9e90 con 0x7f5bd813f840 2023-10-07T07:58:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bddb46700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd80c6660 con 0x7f5bd813fc10 2023-10-07T07:58:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8068d30 0x7f5bd8155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8068d30 0x7f5bd8155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35944/0 (socket says 172.21.15.7:35944) 2023-10-07T07:58:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd77fe700 1 -- 172.21.15.7:0/646358410 learned_addr learned my addr 172.21.15.7:0/646358410 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd6ffd700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd813f840 0x7f5bd80abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd7fff700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 0x7f5bd80ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd77fe700 1 -- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 msgr2=0x7f5bd80ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd77fe700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 0x7f5bd80ac510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd77fe700 1 -- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd813f840 msgr2=0x7f5bd80abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd77fe700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd813f840 0x7f5bd80abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.699+0000 7f5bd77fe700 1 -- 172.21.15.7:0/646358410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc800b660 con 0x7f5bd8068d30 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.700+0000 7f5bd77fe700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8068d30 0x7f5bd8155b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80c7490 tx=0x7f5bc8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.700+0000 7f5bcf7fe700 1 -- 172.21.15.7:0/646358410 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8020070 con 0x7f5bd8068d30 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.700+0000 7f5bcf7fe700 1 -- 172.21.15.7:0/646358410 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc801b320 con 0x7f5bd8068d30 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.700+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd80aa130 con 0x7f5bd8068d30 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.700+0000 7f5bcf7fe700 1 -- 172.21.15.7:0/646358410 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8005c10 con 0x7f5bd8068d30 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.700+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd8156080 con 0x7f5bd8068d30 2023-10-07T07:58:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.701+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd804c3c0 con 0x7f5bd8068d30 2023-10-07T07:58:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.701+0000 7f5bcf7fe700 1 -- 172.21.15.7:0/646358410 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5bc8016030 con 0x7f5bd8068d30 2023-10-07T07:58:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.702+0000 7f5bcf7fe700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5bb8066f30 0x7f5bb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.702+0000 7f5bcf7fe700 1 -- 172.21.15.7:0/646358410 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5bc8014070 con 0x7f5bd8068d30 2023-10-07T07:58:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.702+0000 7f5bd6ffd700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5bb8066f30 0x7f5bb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.702+0000 7f5bd6ffd700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5bb8066f30 0x7f5bb80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80ae550 tx=0x7f5bc0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.706+0000 7f5bcf7fe700 1 -- 172.21.15.7:0/646358410 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bc80527e0 con 0x7f5bd8068d30 2023-10-07T07:58:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.907+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bd80cb960 con 0x7f5bd8068d30 2023-10-07T07:58:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.908+0000 7f5bcf7fe700 1 -- 172.21.15.7:0/646358410 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bc8055e00 con 0x7f5bd8068d30 2023-10-07T07:58:16.909 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:58:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5bb8066f30 msgr2=0x7f5bb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5bb8066f30 0x7f5bb80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80ae550 tx=0x7f5bc0008040 comp rx=0 tx=0).stop 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8068d30 msgr2=0x7f5bd8155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8068d30 0x7f5bd8155b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80c7490 tx=0x7f5bc8019000 comp rx=0 tx=0).stop 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 shutdown_connections 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5bb8066f30 0x7f5bb80693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd813fc10 0x7f5bd80ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8068d30 0x7f5bd8155b40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 --2- 172.21.15.7:0/646358410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd813f840 0x7f5bd80abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 >> 172.21.15.7:0/646358410 conn(0x7f5bd80cbe70 msgr2=0x7f5bd806a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 shutdown_connections 2023-10-07T07:58:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:16.910+0000 7f5bddb46700 1 -- 172.21.15.7:0/646358410 wait complete. 2023-10-07T07:58:16.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T07:58:17.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.060+0000 7fa35d7e2700 1 Processor -- start 2023-10-07T07:58:17.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.060+0000 7fa35d7e2700 1 -- start start 2023-10-07T07:58:17.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.061+0000 7fa35d7e2700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa358140b80 0x7fa35806d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.061+0000 7fa35d7e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3580c64b0 0x7fa35806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.061+0000 7fa35d7e2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3580c7610 con 0x7fa3581407b0 2023-10-07T07:58:17.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.061+0000 7fa35d7e2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35806de00 con 0x7fa3580c64b0 2023-10-07T07:58:17.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.061+0000 7fa35d7e2700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35806f620 con 0x7fa358140b80 2023-10-07T07:58:17.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.061+0000 7fa3577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3580c64b0 0x7fa35806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.061+0000 7fa356ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa3581407b0 0x7fa3581581c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55222/0 (socket says 172.21.15.7:55222) 2023-10-07T07:58:17.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.061+0000 7fa356ffd700 1 -- 172.21.15.7:0/3660062339 learned_addr learned my addr 172.21.15.7:0/3660062339 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.061+0000 7fa3567fc700 1 --2- 172.21.15.7:0/3660062339 >> v2:172.21.15.172:3300/0 conn(0x7fa358140b80 0x7fa35806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa3577fe700 1 -- 172.21.15.7:0/3660062339 >> v2:172.21.15.172:3300/0 conn(0x7fa358140b80 msgr2=0x7fa35806d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa3577fe700 1 --2- 172.21.15.7:0/3660062339 >> v2:172.21.15.172:3300/0 conn(0x7fa358140b80 0x7fa35806d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa3577fe700 1 -- 172.21.15.7:0/3660062339 >> v1:172.21.15.7:6789/0 conn(0x7fa3581407b0 legacy=0x7fa3581581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa3577fe700 1 -- 172.21.15.7:0/3660062339 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3580c6a00 con 0x7fa3580c64b0 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa355ffb700 1 -- 172.21.15.7:0/3660062339 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068477939 0 0) 0x7fa3580c7610 con 0x7fa3581407b0 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa3577fe700 1 --2- 172.21.15.7:0/3660062339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3580c64b0 0x7fa35806d8c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa35806ab60 tx=0x7fa350009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9643e1fd2c467175 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa355ffb700 1 -- 172.21.15.7:0/3660062339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35000e070 con 0x7fa3580c64b0 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa355ffb700 1 -- 172.21.15.7:0/3660062339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35000bb60 con 0x7fa3580c64b0 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa355ffb700 1 -- 172.21.15.7:0/3660062339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350012430 con 0x7fa3580c64b0 2023-10-07T07:58:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.062+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3660062339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3580c64b0 msgr2=0x7fa35806d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.063+0000 7fa35d7e2700 1 --2- 172.21.15.7:0/3660062339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3580c64b0 0x7fa35806d8c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa35806ab60 tx=0x7fa350009780 comp rx=0 tx=0).stop 2023-10-07T07:58:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.063+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3660062339 shutdown_connections 2023-10-07T07:58:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.063+0000 7fa35d7e2700 1 --2- 172.21.15.7:0/3660062339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3580c64b0 0x7fa35806d8c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.063+0000 7fa35d7e2700 1 --2- 172.21.15.7:0/3660062339 >> v2:172.21.15.172:3300/0 conn(0x7fa358140b80 0x7fa35806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.063+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3660062339 >> 172.21.15.7:0/3660062339 conn(0x7fa3580cbe70 msgr2=0x7fa3580cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.063+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3660062339 shutdown_connections 2023-10-07T07:58:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.063+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3660062339 wait complete. 2023-10-07T07:58:17.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.064+0000 7fa35d7e2700 1 Processor -- start 2023-10-07T07:58:17.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.064+0000 7fa35d7e2700 1 -- start start 2023-10-07T07:58:17.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa35d7e2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3580c64b0 0x7fa3580ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa35d7e2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3581407b0 0x7fa358156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa35d7e2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa358140b80 0x7fa3580c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa35d7e2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580c3240 con 0x7fa3581407b0 2023-10-07T07:58:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa35d7e2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580ae510 con 0x7fa3580c64b0 2023-10-07T07:58:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa35d7e2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580ad990 con 0x7fa358140b80 2023-10-07T07:58:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3567fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3581407b0 0x7fa358156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3567fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3581407b0 0x7fa358156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35950/0 (socket says 172.21.15.7:35950) 2023-10-07T07:58:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3567fc700 1 -- 172.21.15.7:0/3965839802 learned_addr learned my addr 172.21.15.7:0/3965839802 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa356ffd700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3580c64b0 0x7fa3580ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3577fe700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa358140b80 0x7fa3580c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3567fc700 1 -- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa358140b80 msgr2=0x7fa3580c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3567fc700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa358140b80 0x7fa3580c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3567fc700 1 -- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3580c64b0 msgr2=0x7fa3580ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3567fc700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3580c64b0 0x7fa3580ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3567fc700 1 -- 172.21.15.7:0/3965839802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa350009a20 con 0x7fa3581407b0 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.065+0000 7fa3567fc700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3581407b0 0x7fa358156890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e900 tx=0x7fa34c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.066+0000 7fa33effd700 1 -- 172.21.15.7:0/3965839802 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c013680 con 0x7fa3581407b0 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.066+0000 7fa33effd700 1 -- 172.21.15.7:0/3965839802 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c0137e0 con 0x7fa3581407b0 2023-10-07T07:58:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.066+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3580adcd0 con 0x7fa3581407b0 2023-10-07T07:58:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.066+0000 7fa33effd700 1 -- 172.21.15.7:0/3965839802 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c013ba0 con 0x7fa3581407b0 2023-10-07T07:58:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.066+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3580ad5e0 con 0x7fa3581407b0 2023-10-07T07:58:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.067+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3580c33b0 con 0x7fa3581407b0 2023-10-07T07:58:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.067+0000 7fa33effd700 1 -- 172.21.15.7:0/3965839802 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa34c0123d0 con 0x7fa3581407b0 2023-10-07T07:58:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.067+0000 7fa33effd700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa344066f30 0x7fa3440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.068+0000 7fa33effd700 1 -- 172.21.15.7:0/3965839802 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa34c084620 con 0x7fa3581407b0 2023-10-07T07:58:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.068+0000 7fa356ffd700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa344066f30 0x7fa3440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.068+0000 7fa356ffd700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa344066f30 0x7fa3440693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa348001360 tx=0x7fa348009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.072+0000 7fa33effd700 1 -- 172.21.15.7:0/3965839802 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa34c051890 con 0x7fa3581407b0 2023-10-07T07:58:17.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.272+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa35804c3c0 con 0x7fa3581407b0 2023-10-07T07:58:17.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.273+0000 7fa33effd700 1 -- 172.21.15.7:0/3965839802 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa34c054eb0 con 0x7fa3581407b0 2023-10-07T07:58:17.274 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-07T07:58:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa344066f30 msgr2=0x7fa3440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa344066f30 0x7fa3440693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa348001360 tx=0x7fa348009040 comp rx=0 tx=0).stop 2023-10-07T07:58:17.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3581407b0 msgr2=0x7fa358156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3581407b0 0x7fa358156890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e900 tx=0x7fa34c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 shutdown_connections 2023-10-07T07:58:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa344066f30 0x7fa3440693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa358140b80 0x7fa3580c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3580c64b0 0x7fa3580ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 --2- 172.21.15.7:0/3965839802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3581407b0 0x7fa358156890 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 >> 172.21.15.7:0/3965839802 conn(0x7fa3580cbe70 msgr2=0x7fa358068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 shutdown_connections 2023-10-07T07:58:17.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.275+0000 7fa35d7e2700 1 -- 172.21.15.7:0/3965839802 wait complete. 2023-10-07T07:58:17.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T07:58:17.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.423+0000 7eff13a37700 1 Processor -- start 2023-10-07T07:58:17.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.424+0000 7eff13a37700 1 -- start start 2023-10-07T07:58:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.424+0000 7eff13a37700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7eff0c068100 0x7eff0c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.424+0000 7eff13a37700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c0affc0 0x7eff0c15d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.424+0000 7eff13a37700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c0b1130 con 0x7eff0c0b39b0 2023-10-07T07:58:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.424+0000 7eff13a37700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0c15a8e0 con 0x7eff0c068100 2023-10-07T07:58:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.424+0000 7eff13a37700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0b03f0 con 0x7eff0c0affc0 2023-10-07T07:58:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7eff117d3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7eff0c0b39b0 0x7eff0c0b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55232/0 (socket says 172.21.15.7:55232) 2023-10-07T07:58:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7eff117d3700 1 -- 172.21.15.7:0/200048076 learned_addr learned my addr 172.21.15.7:0/200048076 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7eff10fd2700 1 --2- 172.21.15.7:0/200048076 >> v2:172.21.15.172:3300/0 conn(0x7eff0c068100 0x7eff0c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7eff11fd4700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c0affc0 0x7eff0c15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725246918 0 0) 0x7eff0c0b1130 con 0x7eff0c0b39b0 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efef8003570 con 0x7eff0c0b39b0 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2114101434 0 0) 0x7efef8003570 con 0x7eff0c0b39b0 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c0affc0 msgr2=0x7eff0c15d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c0affc0 0x7eff0c15d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 >> v2:172.21.15.172:3300/0 conn(0x7eff0c068100 msgr2=0x7eff0c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 --2- 172.21.15.7:0/200048076 >> v2:172.21.15.172:3300/0 conn(0x7eff0c068100 0x7eff0c0afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c159cd0 con 0x7eff0c0b39b0 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7eff00001470 con 0x7eff0c0b39b0 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 >> v1:172.21.15.7:6789/0 conn(0x7eff0c0b39b0 legacy=0x7eff0c0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efef8003a60 0x7efef8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7efef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c0affc0 0x7efef80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c15a8e0 con 0x7efef8004810 2023-10-07T07:58:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0b03f0 con 0x7efef8003a60 2023-10-07T07:58:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.425+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef800ba60 con 0x7eff0c0affc0 2023-10-07T07:58:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7eff10fd2700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7efef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7eff117d3700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efef8003a60 0x7efef8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7eff11fd4700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c0affc0 0x7efef80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7eff10fd2700 1 -- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c0affc0 msgr2=0x7efef80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7eff10fd2700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c0affc0 0x7efef80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7eff10fd2700 1 -- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efef8003a60 msgr2=0x7efef8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7eff10fd2700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efef8003a60 0x7efef8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7eff10fd2700 1 -- 172.21.15.7:0/200048076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef8003570 con 0x7efef8004810 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7eff10fd2700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7efef8004c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff08009c30 tx=0x7eff080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d80deb17949111 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0800e710 con 0x7efef8004810 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0800e870 con 0x7efef8004810 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.426+0000 7efefffff700 1 -- 172.21.15.7:0/200048076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0800ec30 con 0x7efef8004810 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.427+0000 7eff13a37700 1 -- 172.21.15.7:0/200048076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 msgr2=0x7efef8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.427+0000 7eff13a37700 1 --2- 172.21.15.7:0/200048076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7efef8004c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff08009c30 tx=0x7eff080091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.427+0000 7eff11fd4700 1 -- 172.21.15.7:0/200048076 reap_dead start 2023-10-07T07:58:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.427+0000 7eff13a37700 1 -- 172.21.15.7:0/200048076 shutdown_connections 2023-10-07T07:58:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.427+0000 7eff13a37700 1 -- 172.21.15.7:0/200048076 >> 172.21.15.7:0/200048076 conn(0x7eff0c06d350 msgr2=0x7eff0c06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.427+0000 7eff13a37700 1 -- 172.21.15.7:0/200048076 shutdown_connections 2023-10-07T07:58:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.427+0000 7eff13a37700 1 -- 172.21.15.7:0/200048076 wait complete. 2023-10-07T07:58:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.429+0000 7eff13a37700 1 Processor -- start 2023-10-07T07:58:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.429+0000 7eff13a37700 1 -- start start 2023-10-07T07:58:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff13a37700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7eff0c146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff13a37700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff0c151df0 0x7eff0c153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff13a37700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c154d10 0x7eff0c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff13a37700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c1573f0 con 0x7efef8004810 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff13a37700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c1471a0 con 0x7eff0c151df0 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff13a37700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c156ac0 con 0x7eff0c154d10 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff117d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7eff0c146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff117d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7eff0c146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35972/0 (socket says 172.21.15.7:35972) 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff117d3700 1 -- 172.21.15.7:0/2881927316 learned_addr learned my addr 172.21.15.7:0/2881927316 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff10fd2700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff0c151df0 0x7eff0c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff11fd4700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c154d10 0x7eff0c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff117d3700 1 -- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c154d10 msgr2=0x7eff0c155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff117d3700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c154d10 0x7eff0c155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff117d3700 1 -- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff0c151df0 msgr2=0x7eff0c153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff117d3700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff0c151df0 0x7eff0c153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff117d3700 1 -- 172.21.15.7:0/2881927316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c156da0 con 0x7efef8004810 2023-10-07T07:58:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.430+0000 7eff11fd4700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c154d10 0x7eff0c155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.431+0000 7eff117d3700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7eff0c146c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7eff000073f0 tx=0x7eff00001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.431+0000 7efefd7fa700 1 -- 172.21.15.7:0/2881927316 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0000f9a0 con 0x7efef8004810 2023-10-07T07:58:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.431+0000 7efefd7fa700 1 -- 172.21.15.7:0/2881927316 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0000fb20 con 0x7efef8004810 2023-10-07T07:58:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.431+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c157080 con 0x7efef8004810 2023-10-07T07:58:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.431+0000 7efefd7fa700 1 -- 172.21.15.7:0/2881927316 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0000fee0 con 0x7efef8004810 2023-10-07T07:58:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.431+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c156790 con 0x7efef8004810 2023-10-07T07:58:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.432+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c04c3c0 con 0x7efef8004810 2023-10-07T07:58:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.432+0000 7efefd7fa700 1 -- 172.21.15.7:0/2881927316 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff000135c0 con 0x7efef8004810 2023-10-07T07:58:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.433+0000 7efefd7fa700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efef0066f30 0x7efef00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.433+0000 7efefd7fa700 1 -- 172.21.15.7:0/2881927316 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7eff000843e0 con 0x7efef8004810 2023-10-07T07:58:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.433+0000 7eff10fd2700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efef0066f30 0x7efef00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.434+0000 7eff10fd2700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efef0066f30 0x7efef00693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff080091e0 tx=0x7eff0801a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.438+0000 7efefd7fa700 1 -- 172.21.15.7:0/2881927316 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff00051650 con 0x7efef8004810 2023-10-07T07:58:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.639+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff0c159fb0 con 0x7efef8004810 2023-10-07T07:58:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.639+0000 7efefd7fa700 1 -- 172.21.15.7:0/2881927316 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff00054c70 con 0x7efef8004810 2023-10-07T07:58:17.641 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-07T07:58:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efef0066f30 msgr2=0x7efef00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efef0066f30 0x7efef00693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff080091e0 tx=0x7eff0801a000 comp rx=0 tx=0).stop 2023-10-07T07:58:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 msgr2=0x7eff0c146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7eff0c146c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7eff000073f0 tx=0x7eff00001e90 comp rx=0 tx=0).stop 2023-10-07T07:58:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 shutdown_connections 2023-10-07T07:58:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efef0066f30 0x7efef00693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff0c154d10 0x7eff0c155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efef8004810 0x7eff0c146c60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 --2- 172.21.15.7:0/2881927316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff0c151df0 0x7eff0c153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 >> 172.21.15.7:0/2881927316 conn(0x7eff0c06d350 msgr2=0x7eff0c0b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 shutdown_connections 2023-10-07T07:58:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.642+0000 7eff13a37700 1 -- 172.21.15.7:0/2881927316 wait complete. 2023-10-07T07:58:17.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.788+0000 7f4cb1354700 1 Processor -- start 2023-10-07T07:58:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.789+0000 7f4cb1354700 1 -- start start 2023-10-07T07:58:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.789+0000 7f4cb1354700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4cac159380 0x7f4cac0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.789+0000 7f4cb1354700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 0x7f4cac0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.789+0000 7f4cb1354700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cac0d0be0 con 0x7f4cac0d0250 2023-10-07T07:58:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.789+0000 7f4cb1354700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cac0b4820 con 0x7f4cac159380 2023-10-07T07:58:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.789+0000 7f4cb1354700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac0b5000 con 0x7f4cac0b4270 2023-10-07T07:58:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4caa7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4cac0d0250 0x7f4cac0b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55244/0 (socket says 172.21.15.7:55244) 2023-10-07T07:58:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4caa7fc700 1 -- 172.21.15.7:0/3901337669 learned_addr learned my addr 172.21.15.7:0/3901337669 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4cab7fe700 1 --2- 172.21.15.7:0/3901337669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 0x7f4cac0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4caaffd700 1 --2- 172.21.15.7:0/3901337669 >> v2:172.21.15.172:3300/0 conn(0x7f4cac159380 0x7f4cac0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4ca9ffb700 1 -- 172.21.15.7:0/3901337669 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469869484 0 0) 0x7f4cac0d0be0 con 0x7f4cac0d0250 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4ca9ffb700 1 -- 172.21.15.7:0/3901337669 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c94003570 con 0x7f4cac0d0250 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4caaffd700 1 -- 172.21.15.7:0/3901337669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 msgr2=0x7f4cac0d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4caaffd700 1 --2- 172.21.15.7:0/3901337669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 0x7f4cac0d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4ca9ffb700 1 -- 172.21.15.7:0/3901337669 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2191589206 0 0) 0x7f4c94003570 con 0x7f4cac0d0250 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4caaffd700 1 -- 172.21.15.7:0/3901337669 >> v1:172.21.15.7:6789/0 conn(0x7f4cac0d0250 legacy=0x7f4cac0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4caaffd700 1 -- 172.21.15.7:0/3901337669 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac0b52e0 con 0x7f4cac159380 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.790+0000 7f4caaffd700 1 --2- 172.21.15.7:0/3901337669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cac159380 0x7f4cac0b5ae0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0098e0 tx=0x7f4c9c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2d6b76d7225960 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4ca9ffb700 1 -- 172.21.15.7:0/3901337669 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0001470 con 0x7f4cac159380 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4ca9ffb700 1 -- 172.21.15.7:0/3901337669 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9c011420 con 0x7f4cac159380 2023-10-07T07:58:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4ca9ffb700 1 -- 172.21.15.7:0/3901337669 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0117a0 con 0x7f4cac159380 2023-10-07T07:58:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4cb1354700 1 -- 172.21.15.7:0/3901337669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cac159380 msgr2=0x7f4cac0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4cb1354700 1 --2- 172.21.15.7:0/3901337669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cac159380 0x7f4cac0b5ae0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0098e0 tx=0x7f4c9c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4cb1354700 1 -- 172.21.15.7:0/3901337669 shutdown_connections 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4cb1354700 1 --2- 172.21.15.7:0/3901337669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 0x7f4cac0d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4cb1354700 1 --2- 172.21.15.7:0/3901337669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cac159380 0x7f4cac0b5ae0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4cb1354700 1 -- 172.21.15.7:0/3901337669 >> 172.21.15.7:0/3901337669 conn(0x7f4cac067be0 msgr2=0x7f4cac067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4cb1354700 1 -- 172.21.15.7:0/3901337669 shutdown_connections 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.791+0000 7f4cb1354700 1 -- 172.21.15.7:0/3901337669 wait complete. 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.793+0000 7f4cb1354700 1 Processor -- start 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.793+0000 7f4cb1354700 1 -- start start 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.793+0000 7f4cb1354700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 0x7f4cac14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.793+0000 7f4cb1354700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cac0d0250 0x7f4cac15b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.793+0000 7f4cb1354700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac159380 0x7f4cac0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cb1354700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac14d9c0 con 0x7f4cac159380 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cb1354700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac14dca0 con 0x7f4cac0d0250 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cb1354700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac14df80 con 0x7f4cac0b4270 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac159380 0x7f4cac0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac159380 0x7f4cac0b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35974/0 (socket says 172.21.15.7:35974) 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cab7fe700 1 -- 172.21.15.7:0/138955834 learned_addr learned my addr 172.21.15.7:0/138955834 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4caaffd700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 0x7f4cac14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4caa7fc700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cac0d0250 0x7f4cac15b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cab7fe700 1 -- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 msgr2=0x7f4cac14bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cab7fe700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 0x7f4cac14bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cab7fe700 1 -- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cac0d0250 msgr2=0x7f4cac15b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cab7fe700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cac0d0250 0x7f4cac15b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cab7fe700 1 -- 172.21.15.7:0/138955834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c009420 con 0x7f4cac159380 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cab7fe700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac159380 0x7f4cac0b1a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009e90 tx=0x7f4ca400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4c92ffd700 1 -- 172.21.15.7:0/138955834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca400cb70 con 0x7f4cac159380 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4c92ffd700 1 -- 172.21.15.7:0/138955834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca400f040 con 0x7f4cac159380 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cac14e260 con 0x7f4cac159380 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4c92ffd700 1 -- 172.21.15.7:0/138955834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4014480 con 0x7f4cac159380 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.794+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac14c320 con 0x7f4cac159380 2023-10-07T07:58:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.796+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cac0716c0 con 0x7f4cac159380 2023-10-07T07:58:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.796+0000 7f4c92ffd700 1 -- 172.21.15.7:0/138955834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ca4014830 con 0x7f4cac159380 2023-10-07T07:58:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.796+0000 7f4c92ffd700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c98066f30 0x7f4c980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.797+0000 7f4c92ffd700 1 -- 172.21.15.7:0/138955834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ca40851c0 con 0x7f4cac159380 2023-10-07T07:58:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.797+0000 7f4caaffd700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c98066f30 0x7f4c980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:17.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.797+0000 7f4caaffd700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c98066f30 0x7f4c980693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0091e0 tx=0x7f4c9c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:17.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:17.801+0000 7f4c92ffd700 1 -- 172.21.15.7:0/138955834 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ca4052430 con 0x7f4cac159380 2023-10-07T07:58:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.006+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cac0b34d0 con 0x7f4cac159380 2023-10-07T07:58:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.007+0000 7f4c92ffd700 1 -- 172.21.15.7:0/138955834 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f4ca4055a50 con 0x7f4cac159380 2023-10-07T07:58:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:18.009 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c98066f30 msgr2=0x7f4c980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c98066f30 0x7f4c980693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0091e0 tx=0x7f4c9c016000 comp rx=0 tx=0).stop 2023-10-07T07:58:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac159380 msgr2=0x7f4cac0b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac159380 0x7f4cac0b1a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009e90 tx=0x7f4ca400b040 comp rx=0 tx=0).stop 2023-10-07T07:58:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 shutdown_connections 2023-10-07T07:58:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c98066f30 0x7f4c980693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac159380 0x7f4cac0b1a60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cac0b4270 0x7f4cac14bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 --2- 172.21.15.7:0/138955834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cac0d0250 0x7f4cac15b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 >> 172.21.15.7:0/138955834 conn(0x7f4cac067be0 msgr2=0x7f4cac14cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 shutdown_connections 2023-10-07T07:58:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.010+0000 7f4cb1354700 1 -- 172.21.15.7:0/138955834 wait complete. 2023-10-07T07:58:18.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T07:57:48.285077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T07:57:49.111979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T07:58:18.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:58:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.159+0000 7fa6200e3700 1 Processor -- start 2023-10-07T07:58:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.160+0000 7fa6200e3700 1 -- start start 2023-10-07T07:58:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.160+0000 7fa6200e3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa618156f30 0x7fa6180b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.160+0000 7fa6200e3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6180b18d0 con 0x7fa6180b0190 2023-10-07T07:58:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.160+0000 7fa6200e3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6180b0fa0 con 0x7fa6180b22a0 2023-10-07T07:58:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.160+0000 7fa6200e3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6180b1280 con 0x7fa618156f30 2023-10-07T07:58:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61e680700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa6180b0190 0x7fa6180b3060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55256/0 (socket says 172.21.15.7:55256) 2023-10-07T07:58:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61e680700 1 -- 172.21.15.7:0/3163052148 learned_addr learned my addr 172.21.15.7:0/3163052148 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61de7f700 1 --2- 172.21.15.7:0/3163052148 >> v2:172.21.15.172:3300/0 conn(0x7fa618156f30 0x7fa6180b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61ce7d700 1 -- 172.21.15.7:0/3163052148 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276493681 0 0) 0x7fa6180b18d0 con 0x7fa6180b0190 2023-10-07T07:58:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61ce7d700 1 -- 172.21.15.7:0/3163052148 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa604003570 con 0x7fa6180b0190 2023-10-07T07:58:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61ce7d700 1 -- 172.21.15.7:0/3163052148 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220780379 0 0) 0x7fa6180b0fa0 con 0x7fa6180b22a0 2023-10-07T07:58:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61ce7d700 1 -- 172.21.15.7:0/3163052148 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6180b18d0 con 0x7fa6180b22a0 2023-10-07T07:58:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61de7f700 1 -- 172.21.15.7:0/3163052148 >> v1:172.21.15.172:6789/0 conn(0x7fa6180b22a0 legacy=0x7fa6180afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61de7f700 1 -- 172.21.15.7:0/3163052148 >> v1:172.21.15.7:6789/0 conn(0x7fa6180b0190 legacy=0x7fa6180b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61de7f700 1 -- 172.21.15.7:0/3163052148 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6180b1560 con 0x7fa618156f30 2023-10-07T07:58:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.161+0000 7fa61de7f700 1 --2- 172.21.15.7:0/3163052148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa618156f30 0x7fa6180b1d60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa6080098e0 tx=0x7fa6080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf9c3344d165f2ae server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa61ce7d700 1 -- 172.21.15.7:0/3163052148 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608011630 con 0x7fa618156f30 2023-10-07T07:58:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa61ce7d700 1 -- 172.21.15.7:0/3163052148 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa608011790 con 0x7fa618156f30 2023-10-07T07:58:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa61ce7d700 1 -- 172.21.15.7:0/3163052148 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608011b30 con 0x7fa618156f30 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa6200e3700 1 -- 172.21.15.7:0/3163052148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa618156f30 msgr2=0x7fa6180b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa6200e3700 1 --2- 172.21.15.7:0/3163052148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa618156f30 0x7fa6180b1d60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa6080098e0 tx=0x7fa6080091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa6200e3700 1 -- 172.21.15.7:0/3163052148 shutdown_connections 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa6200e3700 1 --2- 172.21.15.7:0/3163052148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa618156f30 0x7fa6180b1d60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa6200e3700 1 -- 172.21.15.7:0/3163052148 >> 172.21.15.7:0/3163052148 conn(0x7fa61806d350 msgr2=0x7fa61806d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa6200e3700 1 -- 172.21.15.7:0/3163052148 shutdown_connections 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.162+0000 7fa6200e3700 1 -- 172.21.15.7:0/3163052148 wait complete. 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.164+0000 7fa6200e3700 1 Processor -- start 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.164+0000 7fa6200e3700 1 -- start start 2023-10-07T07:58:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.164+0000 7fa6200e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6180b0190 0x7fa6181468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.164+0000 7fa6200e3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6180b22a0 0x7fa61814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.164+0000 7fa6200e3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa618156f30 0x7fa6180ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.164+0000 7fa6200e3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61814d280 con 0x7fa6180b0190 2023-10-07T07:58:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.164+0000 7fa6200e3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6181676b0 con 0x7fa6180b22a0 2023-10-07T07:58:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.164+0000 7fa6200e3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61814c670 con 0x7fa618156f30 2023-10-07T07:58:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.164+0000 7fa61de7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6180b0190 0x7fa6181468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61de7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6180b0190 0x7fa6181468a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35982/0 (socket says 172.21.15.7:35982) 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61de7f700 1 -- 172.21.15.7:0/85370327 learned_addr learned my addr 172.21.15.7:0/85370327 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61e680700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa618156f30 0x7fa6180ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61d67e700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6180b22a0 0x7fa61814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61de7f700 1 -- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa618156f30 msgr2=0x7fa6180ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61de7f700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa618156f30 0x7fa6180ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61de7f700 1 -- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6180b22a0 msgr2=0x7fa61814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61de7f700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6180b22a0 0x7fa61814bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61de7f700 1 -- 172.21.15.7:0/85370327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa608009420 con 0x7fa6180b0190 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61d67e700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6180b22a0 0x7fa61814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa61de7f700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6180b0190 0x7fa6181468a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa608009ed0 tx=0x7fa608017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa60dffb700 1 -- 172.21.15.7:0/85370327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60801f070 con 0x7fa6180b0190 2023-10-07T07:58:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa60dffb700 1 -- 172.21.15.7:0/85370327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa608011900 con 0x7fa6180b0190 2023-10-07T07:58:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa60dffb700 1 -- 172.21.15.7:0/85370327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608023400 con 0x7fa6180b0190 2023-10-07T07:58:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.165+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61814c950 con 0x7fa6180b0190 2023-10-07T07:58:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.166+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61814cf10 con 0x7fa6180b0190 2023-10-07T07:58:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.166+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61804c3c0 con 0x7fa6180b0190 2023-10-07T07:58:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.167+0000 7fa60dffb700 1 -- 172.21.15.7:0/85370327 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa608023810 con 0x7fa6180b0190 2023-10-07T07:58:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.167+0000 7fa60dffb700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa5fc066f30 0x7fa5fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.167+0000 7fa60dffb700 1 -- 172.21.15.7:0/85370327 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa6080855c0 con 0x7fa6180b0190 2023-10-07T07:58:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.168+0000 7fa61d67e700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa5fc066f30 0x7fa5fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.168+0000 7fa61d67e700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa5fc066f30 0x7fa5fc0693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6180b2690 tx=0x7fa614009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.172+0000 7fa60dffb700 1 -- 172.21.15.7:0/85370327 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa608052830 con 0x7fa6180b0190 2023-10-07T07:58:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.374+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6181677f0 con 0x7fa6180b0190 2023-10-07T07:58:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.374+0000 7fa60dffb700 1 -- 172.21.15.7:0/85370327 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6181677f0 con 0x7fa6180b0190 2023-10-07T07:58:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa5fc066f30 msgr2=0x7fa5fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa5fc066f30 0x7fa5fc0693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6180b2690 tx=0x7fa614009040 comp rx=0 tx=0).stop 2023-10-07T07:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6180b0190 msgr2=0x7fa6181468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6180b0190 0x7fa6181468a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa608009ed0 tx=0x7fa608017000 comp rx=0 tx=0).stop 2023-10-07T07:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 shutdown_connections 2023-10-07T07:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa5fc066f30 0x7fa5fc0693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa618156f30 0x7fa6180ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6180b0190 0x7fa6181468a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 --2- 172.21.15.7:0/85370327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6180b22a0 0x7fa61814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 >> 172.21.15.7:0/85370327 conn(0x7fa61806d350 msgr2=0x7fa618147560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 shutdown_connections 2023-10-07T07:58:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.377+0000 7fa6200e3700 1 -- 172.21.15.7:0/85370327 wait complete. 2023-10-07T07:58:18.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T07:58:18.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.525+0000 7f2fc852a700 1 Processor -- start 2023-10-07T07:58:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.525+0000 7f2fc852a700 1 -- start start 2023-10-07T07:58:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.526+0000 7f2fc852a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2fc0156f30 0x7f2fc00b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.526+0000 7f2fc852a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2fc00b22a0 0x7f2fc00afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.526+0000 7f2fc852a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc00b18d0 con 0x7f2fc0157300 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.526+0000 7f2fc852a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00b0fa0 con 0x7f2fc00b22a0 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.526+0000 7f2fc852a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00b1280 con 0x7f2fc0156f30 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.526+0000 7f2fc5ac5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2fc00b22a0 0x7f2fc00afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc6ac7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2fc0157300 0x7f2fc00b3060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55266/0 (socket says 172.21.15.7:55266) 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc5ac5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2fc00b22a0 0x7f2fc00afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35988/0 (socket says 172.21.15.7:35988) 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc6ac7700 1 -- 172.21.15.7:0/1847747075 learned_addr learned my addr 172.21.15.7:0/1847747075 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc62c6700 1 --2- 172.21.15.7:0/1847747075 >> v2:172.21.15.172:3300/0 conn(0x7f2fc0156f30 0x7f2fc00b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc5ac5700 1 -- 172.21.15.7:0/1847747075 >> v2:172.21.15.172:3300/0 conn(0x7f2fc0156f30 msgr2=0x7f2fc00b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc5ac5700 1 --2- 172.21.15.7:0/1847747075 >> v2:172.21.15.172:3300/0 conn(0x7f2fc0156f30 0x7f2fc00b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc5ac5700 1 -- 172.21.15.7:0/1847747075 >> v1:172.21.15.7:6789/0 conn(0x7f2fc0157300 legacy=0x7f2fc00b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc5ac5700 1 -- 172.21.15.7:0/1847747075 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00b1560 con 0x7f2fc00b22a0 2023-10-07T07:58:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc5ac5700 1 --2- 172.21.15.7:0/1847747075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc00b22a0 0x7f2fc00afa80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00cf0c0 tx=0x7f2fbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49a62e975417e90 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc52c4700 1 -- 172.21.15.7:0/1847747075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00b6b0 con 0x7f2fc00b22a0 2023-10-07T07:58:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc52c4700 1 -- 172.21.15.7:0/1847747075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc00b810 con 0x7f2fc00b22a0 2023-10-07T07:58:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.527+0000 7f2fc52c4700 1 -- 172.21.15.7:0/1847747075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00bbb0 con 0x7f2fc00b22a0 2023-10-07T07:58:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.528+0000 7f2fc852a700 1 -- 172.21.15.7:0/1847747075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc00b22a0 msgr2=0x7f2fc00afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.528+0000 7f2fc852a700 1 --2- 172.21.15.7:0/1847747075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc00b22a0 0x7f2fc00afa80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00cf0c0 tx=0x7f2fbc0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.528+0000 7f2fc852a700 1 -- 172.21.15.7:0/1847747075 shutdown_connections 2023-10-07T07:58:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.528+0000 7f2fc852a700 1 --2- 172.21.15.7:0/1847747075 >> v2:172.21.15.172:3300/0 conn(0x7f2fc0156f30 0x7f2fc00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.528+0000 7f2fc852a700 1 --2- 172.21.15.7:0/1847747075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc00b22a0 0x7f2fc00afa80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.528+0000 7f2fc852a700 1 -- 172.21.15.7:0/1847747075 >> 172.21.15.7:0/1847747075 conn(0x7f2fc006d350 msgr2=0x7f2fc006d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.528+0000 7f2fc852a700 1 -- 172.21.15.7:0/1847747075 shutdown_connections 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.528+0000 7f2fc852a700 1 -- 172.21.15.7:0/1847747075 wait complete. 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.529+0000 7f2fc852a700 1 Processor -- start 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.529+0000 7f2fc852a700 1 -- start start 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc852a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fc00b22a0 0x7f2fc014dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc852a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fc0156f30 0x7f2fc0157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc852a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc0157300 0x7f2fc00aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc852a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0169900 con 0x7f2fc0157300 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc852a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0169be0 con 0x7f2fc0156f30 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc852a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0169ec0 con 0x7f2fc00b22a0 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc6ac7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc0157300 0x7f2fc00aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc6ac7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc0157300 0x7f2fc00aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35992/0 (socket says 172.21.15.7:35992) 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc6ac7700 1 -- 172.21.15.7:0/3497492116 learned_addr learned my addr 172.21.15.7:0/3497492116 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc62c6700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fc00b22a0 0x7f2fc014dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc5ac5700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fc0156f30 0x7f2fc0157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc6ac7700 1 -- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fc00b22a0 msgr2=0x7f2fc014dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc6ac7700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fc00b22a0 0x7f2fc014dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc6ac7700 1 -- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fc0156f30 msgr2=0x7f2fc0157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc6ac7700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fc0156f30 0x7f2fc0157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.530+0000 7f2fc6ac7700 1 -- 172.21.15.7:0/3497492116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc009420 con 0x7f2fc0157300 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.531+0000 7f2fc6ac7700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc0157300 0x7f2fc00aea50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800cbc0 tx=0x7f2fb80019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.531+0000 7f2fb67fc700 1 -- 172.21.15.7:0/3497492116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb800a940 con 0x7f2fc0157300 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.531+0000 7f2fb67fc700 1 -- 172.21.15.7:0/3497492116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb800aaa0 con 0x7f2fc0157300 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.531+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc016a1a0 con 0x7f2fc0157300 2023-10-07T07:58:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.531+0000 7f2fb67fc700 1 -- 172.21.15.7:0/3497492116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb800ae20 con 0x7f2fc0157300 2023-10-07T07:58:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.531+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc014dff0 con 0x7f2fc0157300 2023-10-07T07:58:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.532+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc016a6c0 con 0x7f2fc0157300 2023-10-07T07:58:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.532+0000 7f2fb67fc700 1 -- 172.21.15.7:0/3497492116 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fb8007300 con 0x7f2fc0157300 2023-10-07T07:58:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.533+0000 7f2fb67fc700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fa4066f30 0x7f2fa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.533+0000 7f2fb67fc700 1 -- 172.21.15.7:0/3497492116 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fb80849d0 con 0x7f2fc0157300 2023-10-07T07:58:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.533+0000 7f2fc62c6700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fa4066f30 0x7f2fa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.533+0000 7f2fc62c6700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fa4066f30 0x7f2fa40693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:18.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.537+0000 7f2fb67fc700 1 -- 172.21.15.7:0/3497492116 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fc016a6c0 con 0x7f2fc0157300 2023-10-07T07:58:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.739+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fc016a6c0 con 0x7f2fc0157300 2023-10-07T07:58:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.740+0000 7f2fb67fc700 1 -- 172.21.15.7:0/3497492116 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fc016a6c0 con 0x7f2fc0157300 2023-10-07T07:58:18.741 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-07T07:58:18.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fa4066f30 msgr2=0x7f2fa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fa4066f30 0x7f2fa40693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0008040 comp rx=0 tx=0).stop 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc0157300 msgr2=0x7f2fc00aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc0157300 0x7f2fc00aea50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800cbc0 tx=0x7f2fb80019c0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 shutdown_connections 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fa4066f30 0x7f2fa40693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc0157300 0x7f2fc00aea50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fc00b22a0 0x7f2fc014dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 --2- 172.21.15.7:0/3497492116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fc0156f30 0x7f2fc0157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 >> 172.21.15.7:0/3497492116 conn(0x7f2fc006d350 msgr2=0x7f2fc0151640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 shutdown_connections 2023-10-07T07:58:18.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.742+0000 7f2fc852a700 1 -- 172.21.15.7:0/3497492116 wait complete. 2023-10-07T07:58:18.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T07:58:18.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.890+0000 7fe472d3c700 1 Processor -- start 2023-10-07T07:58:18.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.890+0000 7fe472d3c700 1 -- start start 2023-10-07T07:58:18.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.891+0000 7fe472d3c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe46c156f30 0x7fe46c0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.891+0000 7fe472d3c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c0b0190 0x7fe46c0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.891+0000 7fe472d3c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c0b18d0 con 0x7fe46c0b22a0 2023-10-07T07:58:18.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.891+0000 7fe472d3c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46c0b0fa0 con 0x7fe46c156f30 2023-10-07T07:58:18.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.891+0000 7fe472d3c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0b1280 con 0x7fe46c0b0190 2023-10-07T07:58:18.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.891+0000 7fe470ad8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe46c156f30 0x7fe46c0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe46bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe46c0b22a0 0x7fe46c0afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55278/0 (socket says 172.21.15.7:55278) 2023-10-07T07:58:18.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe470ad8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe46c156f30 0x7fe46c0b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36006/0 (socket says 172.21.15.7:36006) 2023-10-07T07:58:18.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe46bfff700 1 -- 172.21.15.7:0/2460828757 learned_addr learned my addr 172.21.15.7:0/2460828757 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:18.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe4712d9700 1 --2- 172.21.15.7:0/2460828757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c0b0190 0x7fe46c0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe470ad8700 1 -- 172.21.15.7:0/2460828757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c0b0190 msgr2=0x7fe46c0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:18.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe470ad8700 1 --2- 172.21.15.7:0/2460828757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c0b0190 0x7fe46c0b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe470ad8700 1 -- 172.21.15.7:0/2460828757 >> v1:172.21.15.7:6789/0 conn(0x7fe46c0b22a0 legacy=0x7fe46c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe470ad8700 1 -- 172.21.15.7:0/2460828757 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0b1560 con 0x7fe46c156f30 2023-10-07T07:58:18.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe46b7fe700 1 -- 172.21.15.7:0/2460828757 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375196041 0 0) 0x7fe46c0b18d0 con 0x7fe46c0b22a0 2023-10-07T07:58:18.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe4712d9700 1 --2- 172.21.15.7:0/2460828757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c0b0190 0x7fe46c0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:18.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe470ad8700 1 --2- 172.21.15.7:0/2460828757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c156f30 0x7fe46c0b1d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4600098e0 tx=0x7fe4600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505ef04550665da1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:18.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe46b7fe700 1 -- 172.21.15.7:0/2460828757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460009e80 con 0x7fe46c156f30 2023-10-07T07:58:18.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe46b7fe700 1 -- 172.21.15.7:0/2460828757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46000b860 con 0x7fe46c156f30 2023-10-07T07:58:18.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.892+0000 7fe46b7fe700 1 -- 172.21.15.7:0/2460828757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46000bbe0 con 0x7fe46c156f30 2023-10-07T07:58:18.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.893+0000 7fe472d3c700 1 -- 172.21.15.7:0/2460828757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c156f30 msgr2=0x7fe46c0b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.893+0000 7fe472d3c700 1 --2- 172.21.15.7:0/2460828757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c156f30 0x7fe46c0b1d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4600098e0 tx=0x7fe4600091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.893+0000 7fe472d3c700 1 -- 172.21.15.7:0/2460828757 shutdown_connections 2023-10-07T07:58:18.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.893+0000 7fe472d3c700 1 --2- 172.21.15.7:0/2460828757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c0b0190 0x7fe46c0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.893+0000 7fe472d3c700 1 --2- 172.21.15.7:0/2460828757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c156f30 0x7fe46c0b1d60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.893+0000 7fe472d3c700 1 -- 172.21.15.7:0/2460828757 >> 172.21.15.7:0/2460828757 conn(0x7fe46c06d350 msgr2=0x7fe46c06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:18.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.893+0000 7fe472d3c700 1 -- 172.21.15.7:0/2460828757 shutdown_connections 2023-10-07T07:58:18.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.893+0000 7fe472d3c700 1 -- 172.21.15.7:0/2460828757 wait complete. 2023-10-07T07:58:18.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.894+0000 7fe472d3c700 1 Processor -- start 2023-10-07T07:58:18.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.894+0000 7fe472d3c700 1 -- start start 2023-10-07T07:58:18.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.895+0000 7fe472d3c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c0b0190 0x7fe46c14c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.895+0000 7fe472d3c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe46c0b22a0 0x7fe46c169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.895+0000 7fe472d3c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c156f30 0x7fe46c169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.895+0000 7fe472d3c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c16a3f0 con 0x7fe46c0b0190 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.895+0000 7fe472d3c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c16d200 con 0x7fe46c0b22a0 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.895+0000 7fe472d3c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c14d1e0 con 0x7fe46c156f30 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.895+0000 7fe470ad8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c0b0190 0x7fe46c14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.895+0000 7fe470ad8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c0b0190 0x7fe46c14c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36020/0 (socket says 172.21.15.7:36020) 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.895+0000 7fe470ad8700 1 -- 172.21.15.7:0/3195733961 learned_addr learned my addr 172.21.15.7:0/3195733961 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe46bfff700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe46c0b22a0 0x7fe46c169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe4712d9700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c156f30 0x7fe46c169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe470ad8700 1 -- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c156f30 msgr2=0x7fe46c169a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe470ad8700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c156f30 0x7fe46c169a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe470ad8700 1 -- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe46c0b22a0 msgr2=0x7fe46c169530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe470ad8700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe46c0b22a0 0x7fe46c169530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe470ad8700 1 -- 172.21.15.7:0/3195733961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe460009420 con 0x7fe46c0b0190 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe4712d9700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c156f30 0x7fe46c169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe470ad8700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c0b0190 0x7fe46c14c340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0b0660 tx=0x7fe460017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe468ff9700 1 -- 172.21.15.7:0/3195733961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46000bbe0 con 0x7fe46c0b0190 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe468ff9700 1 -- 172.21.15.7:0/3195733961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe460011420 con 0x7fe46c0b0190 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c16a110 con 0x7fe46c0b0190 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe468ff9700 1 -- 172.21.15.7:0/3195733961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4600117a0 con 0x7fe46c0b0190 2023-10-07T07:58:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.896+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c16ce90 con 0x7fe46c0b0190 2023-10-07T07:58:18.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.897+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c04c3c0 con 0x7fe46c0b0190 2023-10-07T07:58:18.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.898+0000 7fe468ff9700 1 -- 172.21.15.7:0/3195733961 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe460011bd0 con 0x7fe46c0b0190 2023-10-07T07:58:18.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.898+0000 7fe468ff9700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe450066f30 0x7fe4500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:18.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.898+0000 7fe468ff9700 1 -- 172.21.15.7:0/3195733961 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe460085ec0 con 0x7fe46c0b0190 2023-10-07T07:58:18.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.898+0000 7fe46bfff700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe450066f30 0x7fe4500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:18.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.899+0000 7fe46bfff700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe450066f30 0x7fe4500693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe46c151980 tx=0x7fe458009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:18.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:18.903+0000 7fe468ff9700 1 -- 172.21.15.7:0/3195733961 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe460053130 con 0x7fe46c0b0190 2023-10-07T07:58:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.106+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe46c0b3d50 con 0x7fe46c0b0190 2023-10-07T07:58:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.107+0000 7fe468ff9700 1 -- 172.21.15.7:0/3195733961 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe460015020 con 0x7fe46c0b0190 2023-10-07T07:58:19.108 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-07T07:58:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe450066f30 msgr2=0x7fe4500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe450066f30 0x7fe4500693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe46c151980 tx=0x7fe458009040 comp rx=0 tx=0).stop 2023-10-07T07:58:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c0b0190 msgr2=0x7fe46c14c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c0b0190 0x7fe46c14c340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0b0660 tx=0x7fe460017000 comp rx=0 tx=0).stop 2023-10-07T07:58:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 shutdown_connections 2023-10-07T07:58:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe450066f30 0x7fe4500693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe46c156f30 0x7fe46c169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46c0b0190 0x7fe46c14c340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 --2- 172.21.15.7:0/3195733961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe46c0b22a0 0x7fe46c169530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 >> 172.21.15.7:0/3195733961 conn(0x7fe46c06d350 msgr2=0x7fe46c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 shutdown_connections 2023-10-07T07:58:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.109+0000 7fe472d3c700 1 -- 172.21.15.7:0/3195733961 wait complete. 2023-10-07T07:58:19.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.255+0000 7f7059e0e700 1 Processor -- start 2023-10-07T07:58:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.255+0000 7f7059e0e700 1 -- start start 2023-10-07T07:58:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f7059e0e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f705406e9d0 0x7f70540c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f7059e0e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70541562f0 0x7f705406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f7059e0e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705406b700 con 0x7f7054155f20 2023-10-07T07:58:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f7059e0e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705406aaf0 con 0x7f70541562f0 2023-10-07T07:58:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f7059e0e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705406add0 con 0x7f705406e9d0 2023-10-07T07:58:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f70537fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7054155f20 0x7f705406e2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55286/0 (socket says 172.21.15.7:55286) 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f70537fe700 1 -- 172.21.15.7:0/3744393604 learned_addr learned my addr 172.21.15.7:0/3744393604 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f7052ffd700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f705406e9d0 0x7f70540c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f7053fff700 1 --2- 172.21.15.7:0/3744393604 >> v2:172.21.15.172:3300/0 conn(0x7f70541562f0 0x7f705406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305267683 0 0) 0x7f705406b700 con 0x7f7054155f20 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f703c003570 con 0x7f7054155f20 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2999761634 0 0) 0x7f703c003570 con 0x7f7054155f20 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f705406e9d0 msgr2=0x7f70540c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f70527fc700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f705406e9d0 0x7f70540c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 >> v2:172.21.15.172:3300/0 conn(0x7f70541562f0 msgr2=0x7f705406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.256+0000 7f70527fc700 1 --2- 172.21.15.7:0/3744393604 >> v2:172.21.15.172:3300/0 conn(0x7f70541562f0 0x7f705406a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705406b0b0 con 0x7f7054155f20 2023-10-07T07:58:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f7044001470 con 0x7f7054155f20 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 >> v1:172.21.15.7:6789/0 conn(0x7f7054155f20 legacy=0x7f705406e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c003a60 0x7f703c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c004810 0x7f703c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f705406e9d0 0x7f703c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705406aaf0 con 0x7f703c003a60 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705406add0 con 0x7f703c004810 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c00ba60 con 0x7f705406e9d0 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70537fe700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c003a60 0x7f703c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f7053fff700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f705406e9d0 0x7f703c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f7052ffd700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c004810 0x7f703c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70537fe700 1 -- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f705406e9d0 msgr2=0x7f703c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70537fe700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f705406e9d0 0x7f703c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70537fe700 1 -- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c004810 msgr2=0x7f703c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70537fe700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c004810 0x7f703c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70537fe700 1 -- 172.21.15.7:0/3744393604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c003570 con 0x7f703c003a60 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70537fe700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c003a60 0x7f703c003e70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f704400c3f0 tx=0x7f7044001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9285a5e53e88c6ae server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704400cc80 con 0x7f703c003a60 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704400ce20 con 0x7f703c003a60 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.257+0000 7f70527fc700 1 -- 172.21.15.7:0/3744393604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044005b70 con 0x7f703c003a60 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.258+0000 7f7059e0e700 1 -- 172.21.15.7:0/3744393604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c003a60 msgr2=0x7f703c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.258+0000 7f7059e0e700 1 --2- 172.21.15.7:0/3744393604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c003a60 0x7f703c003e70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f704400c3f0 tx=0x7f7044001e90 comp rx=0 tx=0).stop 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.258+0000 7f7053fff700 1 -- 172.21.15.7:0/3744393604 reap_dead start 2023-10-07T07:58:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.258+0000 7f7059e0e700 1 -- 172.21.15.7:0/3744393604 shutdown_connections 2023-10-07T07:58:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.258+0000 7f7059e0e700 1 -- 172.21.15.7:0/3744393604 >> 172.21.15.7:0/3744393604 conn(0x7f7054068bd0 msgr2=0x7f70540ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.258+0000 7f7059e0e700 1 -- 172.21.15.7:0/3744393604 shutdown_connections 2023-10-07T07:58:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.258+0000 7f7059e0e700 1 -- 172.21.15.7:0/3744393604 wait complete. 2023-10-07T07:58:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.259+0000 7f7059e0e700 1 Processor -- start 2023-10-07T07:58:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.259+0000 7f7059e0e700 1 -- start start 2023-10-07T07:58:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7059e0e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c003a60 0x7f7054147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7059e0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70540b2490 0x7f70540b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7059e0e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70540c6070 0x7f70540c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7059e0e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705416a8a0 con 0x7f70540b2490 2023-10-07T07:58:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7059e0e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70540c6a20 con 0x7f703c003a60 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7059e0e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7054169b20 con 0x7f70540c6070 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7052ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70540b2490 0x7f70540b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7052ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70540b2490 0x7f70540b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36030/0 (socket says 172.21.15.7:36030) 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7052ffd700 1 -- 172.21.15.7:0/1139452855 learned_addr learned my addr 172.21.15.7:0/1139452855 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f70537fe700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c003a60 0x7f7054147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.260+0000 7f7053fff700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70540c6070 0x7f70540c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f7052ffd700 1 -- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70540c6070 msgr2=0x7f70540c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f7052ffd700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70540c6070 0x7f70540c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f7052ffd700 1 -- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c003a60 msgr2=0x7f7054147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f7052ffd700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c003a60 0x7f7054147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f7052ffd700 1 -- 172.21.15.7:0/1139452855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054169e00 con 0x7f70540b2490 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f7052ffd700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70540b2490 0x7f70540b2900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7048011d40 tx=0x7f70480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f703b7fe700 1 -- 172.21.15.7:0/1139452855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704800f990 con 0x7f70540b2490 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f703b7fe700 1 -- 172.21.15.7:0/1139452855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704800faf0 con 0x7f70540b2490 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705416a0e0 con 0x7f70540b2490 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f703b7fe700 1 -- 172.21.15.7:0/1139452855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704800fe70 con 0x7f70540b2490 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.261+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7054148050 con 0x7f70540b2490 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.262+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70541523e0 con 0x7f70540b2490 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.263+0000 7f703b7fe700 1 -- 172.21.15.7:0/1139452855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7048014650 con 0x7f70540b2490 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.263+0000 7f703b7fe700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7040066f30 0x7f70400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.263+0000 7f703b7fe700 1 -- 172.21.15.7:0/1139452855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7048018030 con 0x7f70540b2490 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.263+0000 7f70537fe700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7040066f30 0x7f70400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.264+0000 7f70537fe700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7040066f30 0x7f70400693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f704400c3f0 tx=0x7f7044015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:19.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.268+0000 7f703b7fe700 1 -- 172.21.15.7:0/1139452855 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70541523e0 con 0x7f70540b2490 2023-10-07T07:58:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.472+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70541523e0 con 0x7f70540b2490 2023-10-07T07:58:19.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.473+0000 7f703b7fe700 1 -- 172.21.15.7:0/1139452855 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f70541523e0 con 0x7f70540b2490 2023-10-07T07:58:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:19.474 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7040066f30 msgr2=0x7f70400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7040066f30 0x7f70400693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f704400c3f0 tx=0x7f7044015000 comp rx=0 tx=0).stop 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70540b2490 msgr2=0x7f70540b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70540b2490 0x7f70540b2900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7048011d40 tx=0x7f70480091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 shutdown_connections 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7040066f30 0x7f70400693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70540c6070 0x7f70540c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c003a60 0x7f7054147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 --2- 172.21.15.7:0/1139452855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70540b2490 0x7f70540b2900 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.475+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 >> 172.21.15.7:0/1139452855 conn(0x7f7054068bd0 msgr2=0x7f70540c2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.476+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 shutdown_connections 2023-10-07T07:58:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.476+0000 7f7059e0e700 1 -- 172.21.15.7:0/1139452855 wait complete. 2023-10-07T07:58:19.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T07:57:48.285077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T07:57:49.111979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T07:58:19.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:58:19.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.625+0000 7fb21aa92700 1 Processor -- start 2023-10-07T07:58:19.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.625+0000 7fb21aa92700 1 -- start start 2023-10-07T07:58:19.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21aa92700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb214067610 0x7fb214153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21aa92700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2141539c0 0x7fb2140aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21aa92700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21406f320 con 0x7fb214067240 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21aa92700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21406a500 con 0x7fb2141539c0 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21aa92700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb214153db0 con 0x7fb214067610 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21902f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2141539c0 0x7fb2140aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21882e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb214067240 0x7fb21415a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46844/0 (socket says 172.21.15.7:46844) 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21882e700 1 -- 172.21.15.7:0/1607649720 learned_addr learned my addr 172.21.15.7:0/1607649720 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb213fff700 1 --2- 172.21.15.7:0/1607649720 >> v2:172.21.15.172:3300/0 conn(0x7fb214067610 0x7fb214153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21902f700 1 -- 172.21.15.7:0/1607649720 >> v2:172.21.15.172:3300/0 conn(0x7fb214067610 msgr2=0x7fb214153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21902f700 1 --2- 172.21.15.7:0/1607649720 >> v2:172.21.15.172:3300/0 conn(0x7fb214067610 0x7fb214153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21902f700 1 -- 172.21.15.7:0/1607649720 >> v1:172.21.15.7:6789/0 conn(0x7fb214067240 legacy=0x7fb21415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb2137fe700 1 -- 172.21.15.7:0/1607649720 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701716917 0 0) 0x7fb21406f320 con 0x7fb214067240 2023-10-07T07:58:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21902f700 1 -- 172.21.15.7:0/1607649720 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21406e7a0 con 0x7fb2141539c0 2023-10-07T07:58:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.626+0000 7fb21902f700 1 --2- 172.21.15.7:0/1607649720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2141539c0 0x7fb2140aa980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb21406f930 tx=0x7fb20c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5173180ef095fb7b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb2137fe700 1 -- 172.21.15.7:0/1607649720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00e070 con 0x7fb2141539c0 2023-10-07T07:58:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb2137fe700 1 -- 172.21.15.7:0/1607649720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c00bb60 con 0x7fb2141539c0 2023-10-07T07:58:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb2137fe700 1 -- 172.21.15.7:0/1607649720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c012430 con 0x7fb2141539c0 2023-10-07T07:58:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb21aa92700 1 -- 172.21.15.7:0/1607649720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2141539c0 msgr2=0x7fb2140aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb21aa92700 1 --2- 172.21.15.7:0/1607649720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2141539c0 0x7fb2140aa980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb21406f930 tx=0x7fb20c009780 comp rx=0 tx=0).stop 2023-10-07T07:58:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb21aa92700 1 -- 172.21.15.7:0/1607649720 shutdown_connections 2023-10-07T07:58:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb21aa92700 1 --2- 172.21.15.7:0/1607649720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2141539c0 0x7fb2140aa980 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb21aa92700 1 --2- 172.21.15.7:0/1607649720 >> v2:172.21.15.172:3300/0 conn(0x7fb214067610 0x7fb214153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb21aa92700 1 -- 172.21.15.7:0/1607649720 >> 172.21.15.7:0/1607649720 conn(0x7fb214068100 msgr2=0x7fb214068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:19.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb21aa92700 1 -- 172.21.15.7:0/1607649720 shutdown_connections 2023-10-07T07:58:19.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.627+0000 7fb21aa92700 1 -- 172.21.15.7:0/1607649720 wait complete. 2023-10-07T07:58:19.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.629+0000 7fb21aa92700 1 Processor -- start 2023-10-07T07:58:19.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.629+0000 7fb21aa92700 1 -- start start 2023-10-07T07:58:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.630+0000 7fb21aa92700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb214067240 0x7fb21414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.630+0000 7fb21aa92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb214067610 0x7fb214152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.630+0000 7fb21aa92700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2141539c0 0x7fb214152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.630+0000 7fb21aa92700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2140b04a0 con 0x7fb214067610 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.630+0000 7fb21aa92700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2140cc6b0 con 0x7fb2141539c0 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.630+0000 7fb21aa92700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2140cc130 con 0x7fb214067240 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.630+0000 7fb213fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb214067610 0x7fb214152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.630+0000 7fb213fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb214067610 0x7fb214152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46754/0 (socket says 172.21.15.7:46754) 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.630+0000 7fb213fff700 1 -- 172.21.15.7:0/265693172 learned_addr learned my addr 172.21.15.7:0/265693172 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb21902f700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2141539c0 0x7fb214152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb21882e700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb214067240 0x7fb21414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb213fff700 1 -- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb214067240 msgr2=0x7fb21414f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb213fff700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb214067240 0x7fb21414f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb213fff700 1 -- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2141539c0 msgr2=0x7fb214152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb213fff700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2141539c0 0x7fb214152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb213fff700 1 -- 172.21.15.7:0/265693172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c009a20 con 0x7fb214067610 2023-10-07T07:58:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb213fff700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb214067610 0x7fb214152670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb20000ec80 tx=0x7fb2000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb210ff9700 1 -- 172.21.15.7:0/265693172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000c9d0 con 0x7fb214067610 2023-10-07T07:58:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb210ff9700 1 -- 172.21.15.7:0/265693172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20000cb30 con 0x7fb214067610 2023-10-07T07:58:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2140cc3d0 con 0x7fb214067610 2023-10-07T07:58:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb210ff9700 1 -- 172.21.15.7:0/265693172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200013470 con 0x7fb214067610 2023-10-07T07:58:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.631+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2140b0130 con 0x7fb214067610 2023-10-07T07:58:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.632+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21404c3c0 con 0x7fb214067610 2023-10-07T07:58:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.633+0000 7fb210ff9700 1 -- 172.21.15.7:0/265693172 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb200013820 con 0x7fb214067610 2023-10-07T07:58:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.633+0000 7fb210ff9700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb1f8066f30 0x7fb1f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.633+0000 7fb210ff9700 1 -- 172.21.15.7:0/265693172 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb2000848a0 con 0x7fb214067610 2023-10-07T07:58:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.633+0000 7fb21882e700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb1f8066f30 0x7fb1f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.634+0000 7fb21882e700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb1f8066f30 0x7fb1f80693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2140b2e80 tx=0x7fb208009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.638+0000 7fb210ff9700 1 -- 172.21.15.7:0/265693172 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb200051b10 con 0x7fb214067610 2023-10-07T07:58:19.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.837+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb21406ea80 con 0x7fb214067610 2023-10-07T07:58:19.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.838+0000 7fb210ff9700 1 -- 172.21.15.7:0/265693172 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb200055130 con 0x7fb214067610 2023-10-07T07:58:19.839 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:58:19.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb1f8066f30 msgr2=0x7fb1f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb1f8066f30 0x7fb1f80693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2140b2e80 tx=0x7fb208009040 comp rx=0 tx=0).stop 2023-10-07T07:58:19.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb214067610 msgr2=0x7fb214152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb214067610 0x7fb214152670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb20000ec80 tx=0x7fb2000091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 shutdown_connections 2023-10-07T07:58:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb1f8066f30 0x7fb1f80693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2141539c0 0x7fb214152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb214067240 0x7fb21414f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 --2- 172.21.15.7:0/265693172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb214067610 0x7fb214152670 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 >> 172.21.15.7:0/265693172 conn(0x7fb214068100 msgr2=0x7fb2140c7810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 shutdown_connections 2023-10-07T07:58:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.840+0000 7fb21aa92700 1 -- 172.21.15.7:0/265693172 wait complete. 2023-10-07T07:58:19.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T07:58:19.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.989+0000 7f8a5702d700 1 Processor -- start 2023-10-07T07:58:19.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.989+0000 7f8a5702d700 1 -- start start 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a5702d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8a50067610 0x7f8a50153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a5702d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a501539c0 0x7f8a500aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a5702d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5006f320 con 0x7f8a50067240 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a5702d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5006a500 con 0x7f8a501539c0 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a5702d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a50153db0 con 0x7f8a50067610 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a555ca700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a501539c0 0x7f8a500aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a54dc9700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8a50067240 0x7f8a5015a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46848/0 (socket says 172.21.15.7:46848) 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a54dc9700 1 -- 172.21.15.7:0/340259589 learned_addr learned my addr 172.21.15.7:0/340259589 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a4ffff700 1 --2- 172.21.15.7:0/340259589 >> v2:172.21.15.172:3300/0 conn(0x7f8a50067610 0x7f8a50153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a555ca700 1 -- 172.21.15.7:0/340259589 >> v2:172.21.15.172:3300/0 conn(0x7f8a50067610 msgr2=0x7f8a50153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a555ca700 1 --2- 172.21.15.7:0/340259589 >> v2:172.21.15.172:3300/0 conn(0x7f8a50067610 0x7f8a50153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a555ca700 1 -- 172.21.15.7:0/340259589 >> v1:172.21.15.7:6789/0 conn(0x7f8a50067240 legacy=0x7f8a5015a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.990+0000 7f8a555ca700 1 -- 172.21.15.7:0/340259589 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5006e7a0 con 0x7f8a501539c0 2023-10-07T07:58:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a555ca700 1 --2- 172.21.15.7:0/340259589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 0x7f8a500aa980 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a5006f930 tx=0x7f8a48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac9ad4151f7b849 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a4f7fe700 1 -- 172.21.15.7:0/340259589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800e070 con 0x7f8a501539c0 2023-10-07T07:58:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a4f7fe700 1 -- 172.21.15.7:0/340259589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4800bc30 con 0x7f8a501539c0 2023-10-07T07:58:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a4f7fe700 1 -- 172.21.15.7:0/340259589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48012430 con 0x7f8a501539c0 2023-10-07T07:58:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a5702d700 1 -- 172.21.15.7:0/340259589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 msgr2=0x7f8a500aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a5702d700 1 --2- 172.21.15.7:0/340259589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 0x7f8a500aa980 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a5006f930 tx=0x7f8a48009780 comp rx=0 tx=0).stop 2023-10-07T07:58:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a5702d700 1 -- 172.21.15.7:0/340259589 shutdown_connections 2023-10-07T07:58:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a5702d700 1 --2- 172.21.15.7:0/340259589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 0x7f8a500aa980 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a5702d700 1 --2- 172.21.15.7:0/340259589 >> v2:172.21.15.172:3300/0 conn(0x7f8a50067610 0x7f8a50153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a5702d700 1 -- 172.21.15.7:0/340259589 >> 172.21.15.7:0/340259589 conn(0x7f8a50068100 msgr2=0x7f8a50068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:19.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a5702d700 1 -- 172.21.15.7:0/340259589 shutdown_connections 2023-10-07T07:58:19.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.991+0000 7f8a5702d700 1 -- 172.21.15.7:0/340259589 wait complete. 2023-10-07T07:58:19.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.993+0000 7f8a5702d700 1 Processor -- start 2023-10-07T07:58:19.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.993+0000 7f8a5702d700 1 -- start start 2023-10-07T07:58:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.993+0000 7f8a5702d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a50067240 0x7f8a500c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a5702d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a50067610 0x7f8a500b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a5702d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 0x7f8a5014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a5702d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a500b0090 con 0x7f8a501539c0 2023-10-07T07:58:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a5702d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a500c5970 con 0x7f8a50067240 2023-10-07T07:58:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a5702d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a500c5c50 con 0x7f8a50067610 2023-10-07T07:58:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a555ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 0x7f8a5014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a555ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 0x7f8a5014eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46768/0 (socket says 172.21.15.7:46768) 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a555ca700 1 -- 172.21.15.7:0/1332920052 learned_addr learned my addr 172.21.15.7:0/1332920052 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a4ffff700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a50067610 0x7f8a500b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a54dc9700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a50067240 0x7f8a500c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a555ca700 1 -- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a50067610 msgr2=0x7f8a500b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a555ca700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a50067610 0x7f8a500b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a555ca700 1 -- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a50067240 msgr2=0x7f8a500c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a555ca700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a50067240 0x7f8a500c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a555ca700 1 -- 172.21.15.7:0/1332920052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a48009a20 con 0x7f8a501539c0 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a54dc9700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a50067240 0x7f8a500c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a4ffff700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a50067610 0x7f8a500b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a555ca700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 0x7f8a5014eac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8a48013000 tx=0x7f8a48019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a4cff9700 1 -- 172.21.15.7:0/1332920052 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4801e630 con 0x7f8a501539c0 2023-10-07T07:58:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a4cff9700 1 -- 172.21.15.7:0/1332920052 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4801e7f0 con 0x7f8a501539c0 2023-10-07T07:58:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.994+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a500c5f30 con 0x7f8a501539c0 2023-10-07T07:58:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.995+0000 7f8a4cff9700 1 -- 172.21.15.7:0/1332920052 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4801ebb0 con 0x7f8a501539c0 2023-10-07T07:58:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.995+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a500c64f0 con 0x7f8a501539c0 2023-10-07T07:58:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.995+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5004c3c0 con 0x7f8a501539c0 2023-10-07T07:58:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.996+0000 7f8a4cff9700 1 -- 172.21.15.7:0/1332920052 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a48024070 con 0x7f8a501539c0 2023-10-07T07:58:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.996+0000 7f8a4cff9700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8a34066f30 0x7f8a340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.997+0000 7f8a4cff9700 1 -- 172.21.15.7:0/1332920052 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8a48086df0 con 0x7f8a501539c0 2023-10-07T07:58:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.997+0000 7f8a54dc9700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8a34066f30 0x7f8a340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:19.997+0000 7f8a54dc9700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8a34066f30 0x7f8a340693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a440018c0 tx=0x7f8a44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.001+0000 7f8a4cff9700 1 -- 172.21.15.7:0/1332920052 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a48054060 con 0x7f8a501539c0 2023-10-07T07:58:20.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.200+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a5006ea80 con 0x7f8a501539c0 2023-10-07T07:58:20.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.200+0000 7f8a4cff9700 1 -- 172.21.15.7:0/1332920052 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a48057680 con 0x7f8a501539c0 2023-10-07T07:58:20.202 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-07T07:58:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.202+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8a34066f30 msgr2=0x7f8a340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.202+0000 7f8a5702d700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8a34066f30 0x7f8a340693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a440018c0 tx=0x7f8a44009040 comp rx=0 tx=0).stop 2023-10-07T07:58:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.202+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 msgr2=0x7f8a5014eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.202+0000 7f8a5702d700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 0x7f8a5014eac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8a48013000 tx=0x7f8a48019000 comp rx=0 tx=0).stop 2023-10-07T07:58:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.203+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 shutdown_connections 2023-10-07T07:58:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.203+0000 7f8a5702d700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8a34066f30 0x7f8a340693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.203+0000 7f8a5702d700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a501539c0 0x7f8a5014eac0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.203+0000 7f8a5702d700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a50067240 0x7f8a500c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.203+0000 7f8a5702d700 1 --2- 172.21.15.7:0/1332920052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a50067610 0x7f8a500b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.203+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 >> 172.21.15.7:0/1332920052 conn(0x7f8a50068100 msgr2=0x7f8a50151a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.203+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 shutdown_connections 2023-10-07T07:58:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.203+0000 7f8a5702d700 1 -- 172.21.15.7:0/1332920052 wait complete. 2023-10-07T07:58:20.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T07:58:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.352+0000 7f549ac77700 1 Processor -- start 2023-10-07T07:58:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.352+0000 7f549ac77700 1 -- start start 2023-10-07T07:58:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.353+0000 7f549ac77700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5494155d90 0x7f5494156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.353+0000 7f549ac77700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54940c75e0 0x7f549406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.353+0000 7f549ac77700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549406b6f0 con 0x7f54940c7a80 2023-10-07T07:58:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.353+0000 7f549ac77700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54940cbbf0 con 0x7f54940c75e0 2023-10-07T07:58:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.353+0000 7f549ac77700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549406bf70 con 0x7f5494155d90 2023-10-07T07:58:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.353+0000 7f5493fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54940c75e0 0x7f549406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.353+0000 7f5499214700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54940c7a80 0x7f549406c400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46860/0 (socket says 172.21.15.7:46860) 2023-10-07T07:58:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.353+0000 7f5499214700 1 -- 172.21.15.7:0/1366896477 learned_addr learned my addr 172.21.15.7:0/1366896477 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f5498a13700 1 --2- 172.21.15.7:0/1366896477 >> v2:172.21.15.172:3300/0 conn(0x7f5494155d90 0x7f5494156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f54937fe700 1 -- 172.21.15.7:0/1366896477 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607455174 0 0) 0x7f549406b6f0 con 0x7f54940c7a80 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f54937fe700 1 -- 172.21.15.7:0/1366896477 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f547c003570 con 0x7f54940c7a80 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f5493fff700 1 -- 172.21.15.7:0/1366896477 >> v2:172.21.15.172:3300/0 conn(0x7f5494155d90 msgr2=0x7f5494156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f5493fff700 1 --2- 172.21.15.7:0/1366896477 >> v2:172.21.15.172:3300/0 conn(0x7f5494155d90 0x7f5494156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f5493fff700 1 -- 172.21.15.7:0/1366896477 >> v1:172.21.15.7:6789/0 conn(0x7f54940c7a80 legacy=0x7f549406c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f5493fff700 1 -- 172.21.15.7:0/1366896477 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54940cb910 con 0x7f54940c75e0 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f5493fff700 1 --2- 172.21.15.7:0/1366896477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c75e0 0x7f549406e3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f549406f370 tx=0x7f54880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba28c1f0ab2b8bc server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f54937fe700 1 -- 172.21.15.7:0/1366896477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488011630 con 0x7f54940c75e0 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f54937fe700 1 -- 172.21.15.7:0/1366896477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5488011790 con 0x7f54940c75e0 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.354+0000 7f54937fe700 1 -- 172.21.15.7:0/1366896477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488011b30 con 0x7f54940c75e0 2023-10-07T07:58:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.355+0000 7f549ac77700 1 -- 172.21.15.7:0/1366896477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c75e0 msgr2=0x7f549406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.355+0000 7f549ac77700 1 --2- 172.21.15.7:0/1366896477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c75e0 0x7f549406e3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f549406f370 tx=0x7f54880091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.355+0000 7f549ac77700 1 -- 172.21.15.7:0/1366896477 shutdown_connections 2023-10-07T07:58:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.355+0000 7f549ac77700 1 --2- 172.21.15.7:0/1366896477 >> v2:172.21.15.172:3300/0 conn(0x7f5494155d90 0x7f5494156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.355+0000 7f549ac77700 1 --2- 172.21.15.7:0/1366896477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c75e0 0x7f549406e3c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.355+0000 7f549ac77700 1 -- 172.21.15.7:0/1366896477 >> 172.21.15.7:0/1366896477 conn(0x7f5494068100 msgr2=0x7f5494068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.355+0000 7f549ac77700 1 -- 172.21.15.7:0/1366896477 shutdown_connections 2023-10-07T07:58:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.355+0000 7f549ac77700 1 -- 172.21.15.7:0/1366896477 wait complete. 2023-10-07T07:58:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.356+0000 7f549ac77700 1 Processor -- start 2023-10-07T07:58:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.356+0000 7f549ac77700 1 -- start start 2023-10-07T07:58:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f549ac77700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54940c75e0 0x7f549414ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f549ac77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c7a80 0x7f54940cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f549ac77700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5494155d90 0x7f54940cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f549ac77700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54940afd80 con 0x7f54940c7a80 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f549ac77700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54940c44c0 con 0x7f5494155d90 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f549ac77700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54940c47a0 con 0x7f54940c75e0 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5493fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c7a80 0x7f54940cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5493fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c7a80 0x7f54940cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46774/0 (socket says 172.21.15.7:46774) 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5493fff700 1 -- 172.21.15.7:0/1334883503 learned_addr learned my addr 172.21.15.7:0/1334883503 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5499214700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5494155d90 0x7f54940cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5498a13700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54940c75e0 0x7f549414ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5493fff700 1 -- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54940c75e0 msgr2=0x7f549414ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5493fff700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54940c75e0 0x7f549414ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5493fff700 1 -- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5494155d90 msgr2=0x7f54940cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5493fff700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5494155d90 0x7f54940cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.357+0000 7f5493fff700 1 -- 172.21.15.7:0/1334883503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009420 con 0x7f54940c7a80 2023-10-07T07:58:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.358+0000 7f5493fff700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c7a80 0x7f54940cab70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f54940c4000 tx=0x7f5488017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.358+0000 7f5490ff9700 1 -- 172.21.15.7:0/1334883503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488000d10 con 0x7f54940c7a80 2023-10-07T07:58:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.358+0000 7f5490ff9700 1 -- 172.21.15.7:0/1334883503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5488011b10 con 0x7f54940c7a80 2023-10-07T07:58:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.358+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54940c4a80 con 0x7f54940c7a80 2023-10-07T07:58:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.358+0000 7f5490ff9700 1 -- 172.21.15.7:0/1334883503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488023620 con 0x7f54940c7a80 2023-10-07T07:58:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.358+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494150460 con 0x7f54940c7a80 2023-10-07T07:58:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.359+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54940afef0 con 0x7f54940c7a80 2023-10-07T07:58:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.359+0000 7f5490ff9700 1 -- 172.21.15.7:0/1334883503 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54880193b0 con 0x7f54940c7a80 2023-10-07T07:58:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.360+0000 7f5490ff9700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5480066f60 0x7f5480069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.360+0000 7f5490ff9700 1 -- 172.21.15.7:0/1334883503 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5488085a30 con 0x7f54940c7a80 2023-10-07T07:58:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.360+0000 7f5498a13700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5480066f60 0x7f5480069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.361+0000 7f5498a13700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5480066f60 0x7f5480069410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5484006fd0 tx=0x7f5484008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.365+0000 7f5490ff9700 1 -- 172.21.15.7:0/1334883503 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5488052d20 con 0x7f54940c7a80 2023-10-07T07:58:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.568+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f549404c3c0 con 0x7f54940c7a80 2023-10-07T07:58:20.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.568+0000 7f5490ff9700 1 -- 172.21.15.7:0/1334883503 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54880150a0 con 0x7f54940c7a80 2023-10-07T07:58:20.569 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-07T07:58:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5480066f60 msgr2=0x7f5480069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5480066f60 0x7f5480069410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5484006fd0 tx=0x7f5484008040 comp rx=0 tx=0).stop 2023-10-07T07:58:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c7a80 msgr2=0x7f54940cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c7a80 0x7f54940cab70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f54940c4000 tx=0x7f5488017000 comp rx=0 tx=0).stop 2023-10-07T07:58:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 shutdown_connections 2023-10-07T07:58:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5480066f60 0x7f5480069410 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5494155d90 0x7f54940cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54940c75e0 0x7f549414ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 --2- 172.21.15.7:0/1334883503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54940c7a80 0x7f54940cab70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 >> 172.21.15.7:0/1334883503 conn(0x7f5494068100 msgr2=0x7f54940684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.570+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 shutdown_connections 2023-10-07T07:58:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.571+0000 7f549ac77700 1 -- 172.21.15.7:0/1334883503 wait complete. 2023-10-07T07:58:20.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:20.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.716+0000 7fea4cb79700 1 Processor -- start 2023-10-07T07:58:20.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.716+0000 7fea4cb79700 1 -- start start 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.717+0000 7fea4cb79700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea48063230 0x7fea4806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.717+0000 7fea4cb79700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480c9ec0 0x7fea4806e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.717+0000 7fea4cb79700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4806e590 con 0x7fea480c9af0 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.717+0000 7fea4cb79700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea480b03a0 con 0x7fea48063230 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.717+0000 7fea4cb79700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4815a330 con 0x7fea480c9ec0 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.717+0000 7fea3dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea48063230 0x7fea4806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.717+0000 7fea4659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fea480c9af0 0x7fea4815d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46866/0 (socket says 172.21.15.7:46866) 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.717+0000 7fea4659c700 1 -- 172.21.15.7:0/3577364857 learned_addr learned my addr 172.21.15.7:0/3577364857 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.717+0000 7fea46d9d700 1 --2- 172.21.15.7:0/3577364857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480c9ec0 0x7fea4806e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea3dd9b700 1 -- 172.21.15.7:0/3577364857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480c9ec0 msgr2=0x7fea4806e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea3dd9b700 1 --2- 172.21.15.7:0/3577364857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480c9ec0 0x7fea4806e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea3dd9b700 1 -- 172.21.15.7:0/3577364857 >> v1:172.21.15.7:6789/0 conn(0x7fea480c9af0 legacy=0x7fea4815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea3dd9b700 1 -- 172.21.15.7:0/3577364857 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4815a610 con 0x7fea48063230 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea45d9b700 1 -- 172.21.15.7:0/3577364857 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255448935 0 0) 0x7fea4806e590 con 0x7fea480c9af0 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea46d9d700 1 --2- 172.21.15.7:0/3577364857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480c9ec0 0x7fea4806e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea3dd9b700 1 --2- 172.21.15.7:0/3577364857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea48063230 0x7fea4806db10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea300098e0 tx=0x7fea300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b400c5ea70ea9f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea45d9b700 1 -- 172.21.15.7:0/3577364857 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30009e80 con 0x7fea48063230 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea45d9b700 1 -- 172.21.15.7:0/3577364857 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3000b860 con 0x7fea48063230 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.718+0000 7fea45d9b700 1 -- 172.21.15.7:0/3577364857 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000bbe0 con 0x7fea48063230 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.719+0000 7fea4cb79700 1 -- 172.21.15.7:0/3577364857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea48063230 msgr2=0x7fea4806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.719+0000 7fea4cb79700 1 --2- 172.21.15.7:0/3577364857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea48063230 0x7fea4806db10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea300098e0 tx=0x7fea300091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.719+0000 7fea4cb79700 1 -- 172.21.15.7:0/3577364857 shutdown_connections 2023-10-07T07:58:20.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.719+0000 7fea4cb79700 1 --2- 172.21.15.7:0/3577364857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea480c9ec0 0x7fea4806e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.719+0000 7fea4cb79700 1 --2- 172.21.15.7:0/3577364857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea48063230 0x7fea4806db10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.719+0000 7fea4cb79700 1 -- 172.21.15.7:0/3577364857 >> 172.21.15.7:0/3577364857 conn(0x7fea480cd1a0 msgr2=0x7fea480c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:20.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.719+0000 7fea4cb79700 1 -- 172.21.15.7:0/3577364857 shutdown_connections 2023-10-07T07:58:20.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.719+0000 7fea4cb79700 1 -- 172.21.15.7:0/3577364857 wait complete. 2023-10-07T07:58:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.720+0000 7fea4cb79700 1 Processor -- start 2023-10-07T07:58:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.721+0000 7fea4cb79700 1 -- start start 2023-10-07T07:58:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea4cb79700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea48063230 0x7fea4814c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea4cb79700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea480c9af0 0x7fea4815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea4cb79700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea480c9ec0 0x7fea4815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea4cb79700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4815e520 con 0x7fea480c9af0 2023-10-07T07:58:20.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea4cb79700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4814f180 con 0x7fea480c9ec0 2023-10-07T07:58:20.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea4cb79700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea48155b60 con 0x7fea48063230 2023-10-07T07:58:20.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea480c9af0 0x7fea4815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea480c9af0 0x7fea4815b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46798/0 (socket says 172.21.15.7:46798) 2023-10-07T07:58:20.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3dd9b700 1 -- 172.21.15.7:0/2872971045 learned_addr learned my addr 172.21.15.7:0/2872971045 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:20.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea4659c700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea48063230 0x7fea4814c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea46d9d700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea480c9ec0 0x7fea4815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3dd9b700 1 -- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea48063230 msgr2=0x7fea4814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3dd9b700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea48063230 0x7fea4814c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3dd9b700 1 -- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea480c9ec0 msgr2=0x7fea4815bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3dd9b700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea480c9ec0 0x7fea4815bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3dd9b700 1 -- 172.21.15.7:0/2872971045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea30009420 con 0x7fea480c9af0 2023-10-07T07:58:20.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea4659c700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea48063230 0x7fea4814c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:20.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3dd9b700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea480c9af0 0x7fea4815b590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fea4806d7c0 tx=0x7fea30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:20.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3effd700 1 -- 172.21.15.7:0/2872971045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000bbe0 con 0x7fea480c9af0 2023-10-07T07:58:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3effd700 1 -- 172.21.15.7:0/2872971045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea30011420 con 0x7fea480c9af0 2023-10-07T07:58:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea48155e40 con 0x7fea480c9af0 2023-10-07T07:58:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.722+0000 7fea3effd700 1 -- 172.21.15.7:0/2872971045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300117c0 con 0x7fea480c9af0 2023-10-07T07:58:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.723+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4815e1b0 con 0x7fea480c9af0 2023-10-07T07:58:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.724+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea480716c0 con 0x7fea480c9af0 2023-10-07T07:58:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.724+0000 7fea3effd700 1 -- 172.21.15.7:0/2872971045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea30011bb0 con 0x7fea480c9af0 2023-10-07T07:58:20.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.725+0000 7fea3effd700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fea28066f30 0x7fea280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:20.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.725+0000 7fea3effd700 1 -- 172.21.15.7:0/2872971045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fea30085c20 con 0x7fea480c9af0 2023-10-07T07:58:20.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.725+0000 7fea4659c700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fea28066f30 0x7fea280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:20.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.725+0000 7fea4659c700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fea28066f30 0x7fea280693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea38007fd0 tx=0x7fea38009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:20.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.729+0000 7fea3effd700 1 -- 172.21.15.7:0/2872971045 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea30052e90 con 0x7fea480c9af0 2023-10-07T07:58:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.932+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea4815e660 con 0x7fea480c9af0 2023-10-07T07:58:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.933+0000 7fea3effd700 1 -- 172.21.15.7:0/2872971045 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fea30071c40 con 0x7fea480c9af0 2023-10-07T07:58:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:20.935 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fea28066f30 msgr2=0x7fea280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fea28066f30 0x7fea280693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea38007fd0 tx=0x7fea38009040 comp rx=0 tx=0).stop 2023-10-07T07:58:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea480c9af0 msgr2=0x7fea4815b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea480c9af0 0x7fea4815b590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fea4806d7c0 tx=0x7fea30017000 comp rx=0 tx=0).stop 2023-10-07T07:58:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 shutdown_connections 2023-10-07T07:58:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fea28066f30 0x7fea280693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea480c9ec0 0x7fea4815bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea48063230 0x7fea4814c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 --2- 172.21.15.7:0/2872971045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea480c9af0 0x7fea4815b590 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 >> 172.21.15.7:0/2872971045 conn(0x7fea480cd1a0 msgr2=0x7fea4806db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 shutdown_connections 2023-10-07T07:58:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:20.935+0000 7fea4cb79700 1 -- 172.21.15.7:0/2872971045 wait complete. 2023-10-07T07:58:20.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T07:57:48.285077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T07:57:49.111979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T07:58:20.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:58:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.084+0000 7fe5c424d700 1 Processor -- start 2023-10-07T07:58:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.085+0000 7fe5c424d700 1 -- start start 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c424d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc15c920 0x7fe5bc0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c424d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe5bc068710 0x7fe5bc06c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c424d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc06c600 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c424d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5bc0cb3c0 con 0x7fe5bc068710 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c424d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc0cab00 con 0x7fe5bc15c920 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c17e8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe5bc15ccf0 0x7fe5bc068000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46874/0 (socket says 172.21.15.7:46874) 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c17e8700 1 -- 172.21.15.7:0/4054474837 learned_addr learned my addr 172.21.15.7:0/4054474837 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c1fe9700 1 --2- 172.21.15.7:0/4054474837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc15c920 0x7fe5bc0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c27ea700 1 --2- 172.21.15.7:0/4054474837 >> v2:172.21.15.172:3300/0 conn(0x7fe5bc068710 0x7fe5bc06c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c0fe7700 1 -- 172.21.15.7:0/4054474837 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402286082 0 0) 0x7fe5b0001110 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c0fe7700 1 -- 172.21.15.7:0/4054474837 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5a8003570 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c1fe9700 1 -- 172.21.15.7:0/4054474837 >> v2:172.21.15.172:3300/0 conn(0x7fe5bc068710 msgr2=0x7fe5bc06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c1fe9700 1 --2- 172.21.15.7:0/4054474837 >> v2:172.21.15.172:3300/0 conn(0x7fe5bc068710 0x7fe5bc06c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c1fe9700 1 -- 172.21.15.7:0/4054474837 >> v1:172.21.15.7:6789/0 conn(0x7fe5bc15ccf0 legacy=0x7fe5bc068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c0fe7700 1 -- 172.21.15.7:0/4054474837 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570618455 0 0) 0x7fe5a8003570 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.086+0000 7fe5c1fe9700 1 -- 172.21.15.7:0/4054474837 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc06a7a0 con 0x7fe5bc15c920 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c1fe9700 1 --2- 172.21.15.7:0/4054474837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc15c920 0x7fe5bc0ca520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80098d0 tx=0x7fe5b8009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9cde1789020136 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c0fe7700 1 -- 172.21.15.7:0/4054474837 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0001600 con 0x7fe5bc15c920 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c0fe7700 1 -- 172.21.15.7:0/4054474837 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8011420 con 0x7fe5bc15c920 2023-10-07T07:58:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c0fe7700 1 -- 172.21.15.7:0/4054474837 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80117e0 con 0x7fe5bc15c920 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c424d700 1 -- 172.21.15.7:0/4054474837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc15c920 msgr2=0x7fe5bc0ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c424d700 1 --2- 172.21.15.7:0/4054474837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc15c920 0x7fe5bc0ca520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80098d0 tx=0x7fe5b8009120 comp rx=0 tx=0).stop 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c424d700 1 -- 172.21.15.7:0/4054474837 shutdown_connections 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c424d700 1 --2- 172.21.15.7:0/4054474837 >> v2:172.21.15.172:3300/0 conn(0x7fe5bc068710 0x7fe5bc06c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c424d700 1 --2- 172.21.15.7:0/4054474837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc15c920 0x7fe5bc0ca520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.087+0000 7fe5c424d700 1 -- 172.21.15.7:0/4054474837 >> 172.21.15.7:0/4054474837 conn(0x7fe5bc0c9c30 msgr2=0x7fe5bc06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.088+0000 7fe5c424d700 1 -- 172.21.15.7:0/4054474837 shutdown_connections 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.088+0000 7fe5c424d700 1 -- 172.21.15.7:0/4054474837 wait complete. 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.089+0000 7fe5c424d700 1 Processor -- start 2023-10-07T07:58:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.089+0000 7fe5c424d700 1 -- start start 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c424d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc068710 0x7fe5bc0afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c424d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5bc15c920 0x7fe5bc066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c424d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5bc15ccf0 0x7fe5bc067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c424d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc150bc0 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c424d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc14ffb0 con 0x7fe5bc15c920 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c424d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc150290 con 0x7fe5bc068710 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c27ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5bc15ccf0 0x7fe5bc067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c27ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5bc15ccf0 0x7fe5bc067530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46802/0 (socket says 172.21.15.7:46802) 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c27ea700 1 -- 172.21.15.7:0/1021721100 learned_addr learned my addr 172.21.15.7:0/1021721100 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c1fe9700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc068710 0x7fe5bc0afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c17e8700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5bc15c920 0x7fe5bc066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c27ea700 1 -- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc068710 msgr2=0x7fe5bc0afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c27ea700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc068710 0x7fe5bc0afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c27ea700 1 -- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5bc15c920 msgr2=0x7fe5bc066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c27ea700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5bc15c920 0x7fe5bc066ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c27ea700 1 -- 172.21.15.7:0/1021721100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b80093c0 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c1fe9700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc068710 0x7fe5bc0afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c17e8700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5bc15c920 0x7fe5bc066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c27ea700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5bc15ccf0 0x7fe5bc067530 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009e90 tx=0x7fe5b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5adffb700 1 -- 172.21.15.7:0/1021721100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400cc20 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5adffb700 1 -- 172.21.15.7:0/1021721100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b400f040 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5bc150570 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.090+0000 7fe5adffb700 1 -- 172.21.15.7:0/1021721100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4014480 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.091+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc0b03d0 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.092+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5bc150d70 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.092+0000 7fe5adffb700 1 -- 172.21.15.7:0/1021721100 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5b4014830 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.092+0000 7fe5adffb700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5a0066f00 0x7fe5a00693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.093+0000 7fe5adffb700 1 -- 172.21.15.7:0/1021721100 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe5b4084f10 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.093+0000 7fe5c1fe9700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5a0066f00 0x7fe5a00693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.093+0000 7fe5c1fe9700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5a0066f00 0x7fe5a00693b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009390 tx=0x7fe5b8016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.097+0000 7fe5adffb700 1 -- 172.21.15.7:0/1021721100 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5b4055c10 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.296+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5bc04c3c0 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.297+0000 7fe5adffb700 1 -- 172.21.15.7:0/1021721100 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5b40557a0 con 0x7fe5bc15ccf0 2023-10-07T07:58:21.298 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:58:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5a0066f00 msgr2=0x7fe5a00693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5a0066f00 0x7fe5a00693b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009390 tx=0x7fe5b8016000 comp rx=0 tx=0).stop 2023-10-07T07:58:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5bc15ccf0 msgr2=0x7fe5bc067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5bc15ccf0 0x7fe5bc067530 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009e90 tx=0x7fe5b400b040 comp rx=0 tx=0).stop 2023-10-07T07:58:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 shutdown_connections 2023-10-07T07:58:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5a0066f00 0x7fe5a00693b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5bc15ccf0 0x7fe5bc067530 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5bc068710 0x7fe5bc0afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 --2- 172.21.15.7:0/1021721100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5bc15c920 0x7fe5bc066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 >> 172.21.15.7:0/1021721100 conn(0x7fe5bc0c9c30 msgr2=0x7fe5bc0c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 shutdown_connections 2023-10-07T07:58:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.299+0000 7fe5c424d700 1 -- 172.21.15.7:0/1021721100 wait complete. 2023-10-07T07:58:21.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T07:58:21.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.448+0000 7f0819815700 1 Processor -- start 2023-10-07T07:58:21.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.448+0000 7f0819815700 1 -- start start 2023-10-07T07:58:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.449+0000 7f0819815700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0814154ca0 0x7f08140ad990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.449+0000 7f0819815700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081415ae10 con 0x7f081406b610 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.449+0000 7f0819815700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081406d200 con 0x7f081406cd70 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.449+0000 7f0819815700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08140cede0 con 0x7f0814154ca0 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.449+0000 7f0812ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0814154ca0 0x7f08140ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.449+0000 7f08137fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f081406b610 0x7f081406ba10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46884/0 (socket says 172.21.15.7:46884) 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.449+0000 7f08137fe700 1 -- 172.21.15.7:0/851630007 learned_addr learned my addr 172.21.15.7:0/851630007 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.449+0000 7f0812ffd700 1 --2- 172.21.15.7:0/851630007 >> v2:172.21.15.7:3300/0 conn(0x7f0814154ca0 0x7f08140ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46812/0 (socket says 172.21.15.7:46812) 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.450+0000 7f0812ffd700 1 -- 172.21.15.7:0/851630007 >> v1:172.21.15.172:6789/0 conn(0x7f081406cd70 legacy=0x7f0814157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.450+0000 7f0812ffd700 1 -- 172.21.15.7:0/851630007 >> v1:172.21.15.7:6789/0 conn(0x7f081406b610 legacy=0x7f081406ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.450+0000 7f0811ffb700 1 -- 172.21.15.7:0/851630007 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141128017 0 0) 0x7f081415ae10 con 0x7f081406b610 2023-10-07T07:58:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.450+0000 7f0812ffd700 1 -- 172.21.15.7:0/851630007 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081406d4e0 con 0x7f0814154ca0 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.450+0000 7f0812ffd700 1 --2- 172.21.15.7:0/851630007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0814154ca0 0x7f08140ad990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0098e0 tx=0x7f07fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff126fbd2a861d0e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.450+0000 7f0811ffb700 1 -- 172.21.15.7:0/851630007 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc009e80 con 0x7f0814154ca0 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.450+0000 7f0811ffb700 1 -- 172.21.15.7:0/851630007 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc00b860 con 0x7f0814154ca0 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.450+0000 7f0811ffb700 1 -- 172.21.15.7:0/851630007 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00bbe0 con 0x7f0814154ca0 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.451+0000 7f0819815700 1 -- 172.21.15.7:0/851630007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0814154ca0 msgr2=0x7f08140ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.451+0000 7f0819815700 1 --2- 172.21.15.7:0/851630007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0814154ca0 0x7f08140ad990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0098e0 tx=0x7f07fc0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.451+0000 7f0819815700 1 -- 172.21.15.7:0/851630007 shutdown_connections 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.451+0000 7f0819815700 1 --2- 172.21.15.7:0/851630007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0814154ca0 0x7f08140ad990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.451+0000 7f0819815700 1 -- 172.21.15.7:0/851630007 >> 172.21.15.7:0/851630007 conn(0x7f0814067ac0 msgr2=0x7f08140ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.451+0000 7f0819815700 1 -- 172.21.15.7:0/851630007 shutdown_connections 2023-10-07T07:58:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.451+0000 7f0819815700 1 -- 172.21.15.7:0/851630007 wait complete. 2023-10-07T07:58:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.452+0000 7f0819815700 1 Processor -- start 2023-10-07T07:58:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.452+0000 7f0819815700 1 -- start start 2023-10-07T07:58:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f0819815700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f081406b610 0x7f08140c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f0819815700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081406cd70 0x7f08140c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f0819815700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0814154ca0 0x7f08140c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f0819815700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814156d20 con 0x7f081406cd70 2023-10-07T07:58:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f0819815700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814156110 con 0x7f081406b610 2023-10-07T07:58:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f0819815700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08141563f0 con 0x7f0814154ca0 2023-10-07T07:58:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08127fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081406cd70 0x7f08140c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08127fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081406cd70 0x7f08140c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46822/0 (socket says 172.21.15.7:46822) 2023-10-07T07:58:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08127fc700 1 -- 172.21.15.7:0/2688045862 learned_addr learned my addr 172.21.15.7:0/2688045862 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f0812ffd700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f081406b610 0x7f08140c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08137fe700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0814154ca0 0x7f08140c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08127fc700 1 -- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0814154ca0 msgr2=0x7f08140c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08127fc700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0814154ca0 0x7f08140c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08127fc700 1 -- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f081406b610 msgr2=0x7f08140c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08127fc700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f081406b610 0x7f08140c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08127fc700 1 -- 172.21.15.7:0/2688045862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc009420 con 0x7f081406cd70 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08137fe700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0814154ca0 0x7f08140c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.453+0000 7f08127fc700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081406cd70 0x7f08140c6c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f080400c790 tx=0x7f0804007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.454+0000 7f080affd700 1 -- 172.21.15.7:0/2688045862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804011590 con 0x7f081406cd70 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.454+0000 7f080affd700 1 -- 172.21.15.7:0/2688045862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08040116f0 con 0x7f081406cd70 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.454+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08141566d0 con 0x7f081406cd70 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.454+0000 7f080affd700 1 -- 172.21.15.7:0/2688045862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804011ab0 con 0x7f081406cd70 2023-10-07T07:58:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.454+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0814154a40 con 0x7f081406cd70 2023-10-07T07:58:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.455+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0814156e90 con 0x7f081406cd70 2023-10-07T07:58:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.455+0000 7f080affd700 1 -- 172.21.15.7:0/2688045862 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f080401b3b0 con 0x7f081406cd70 2023-10-07T07:58:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.456+0000 7f080affd700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f07f4066f30 0x7f07f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.456+0000 7f080affd700 1 -- 172.21.15.7:0/2688045862 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f08040839c0 con 0x7f081406cd70 2023-10-07T07:58:21.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.456+0000 7f0812ffd700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f07f4066f30 0x7f07f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.457+0000 7f0812ffd700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f07f4066f30 0x7f07f40693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0093f0 tx=0x7f07fc017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.460+0000 7f080affd700 1 -- 172.21.15.7:0/2688045862 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0804050c30 con 0x7f081406cd70 2023-10-07T07:58:21.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.663+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f081404c3c0 con 0x7f081406cd70 2023-10-07T07:58:21.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.663+0000 7f080affd700 1 -- 172.21.15.7:0/2688045862 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08040170a0 con 0x7f081406cd70 2023-10-07T07:58:21.665 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-07T07:58:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.665+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f07f4066f30 msgr2=0x7f07f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.665+0000 7f0819815700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f07f4066f30 0x7f07f40693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0093f0 tx=0x7f07fc017000 comp rx=0 tx=0).stop 2023-10-07T07:58:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.665+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081406cd70 msgr2=0x7f08140c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.665+0000 7f0819815700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081406cd70 0x7f08140c6c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f080400c790 tx=0x7f0804007f20 comp rx=0 tx=0).stop 2023-10-07T07:58:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.666+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 shutdown_connections 2023-10-07T07:58:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.666+0000 7f0819815700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f07f4066f30 0x7f07f40693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.666+0000 7f0819815700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0814154ca0 0x7f08140c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.666+0000 7f0819815700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f081406b610 0x7f08140c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.666+0000 7f0819815700 1 --2- 172.21.15.7:0/2688045862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081406cd70 0x7f08140c6c80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.666+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 >> 172.21.15.7:0/2688045862 conn(0x7f0814067ac0 msgr2=0x7f08140cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.666+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 shutdown_connections 2023-10-07T07:58:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.666+0000 7f0819815700 1 -- 172.21.15.7:0/2688045862 wait complete. 2023-10-07T07:58:21.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T07:58:21.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.818+0000 7f6d50542700 1 Processor -- start 2023-10-07T07:58:21.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.818+0000 7f6d50542700 1 -- start start 2023-10-07T07:58:21.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d50542700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6d48156f30 0x7f6d480b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d50542700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6d480b22a0 0x7f6d480afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d50542700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d480b18d0 con 0x7f6d48157300 2023-10-07T07:58:21.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d50542700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d480b0fa0 con 0x7f6d480b22a0 2023-10-07T07:58:21.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d50542700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d480b1280 con 0x7f6d48156f30 2023-10-07T07:58:21.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d4dadd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6d480b22a0 0x7f6d480afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d4dadd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6d480b22a0 0x7f6d480afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46838/0 (socket says 172.21.15.7:46838) 2023-10-07T07:58:21.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d4dadd700 1 -- 172.21.15.7:0/612991195 learned_addr learned my addr 172.21.15.7:0/612991195 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:21.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d4e2de700 1 --2- 172.21.15.7:0/612991195 >> v2:172.21.15.172:3300/0 conn(0x7f6d48156f30 0x7f6d480b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d4dadd700 1 -- 172.21.15.7:0/612991195 >> v2:172.21.15.172:3300/0 conn(0x7f6d48156f30 msgr2=0x7f6d480b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:21.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d4dadd700 1 --2- 172.21.15.7:0/612991195 >> v2:172.21.15.172:3300/0 conn(0x7f6d48156f30 0x7f6d480b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d4dadd700 1 -- 172.21.15.7:0/612991195 >> v1:172.21.15.172:6789/0 conn(0x7f6d48157300 legacy=0x7f6d480b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:21.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d4dadd700 1 -- 172.21.15.7:0/612991195 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d480b1560 con 0x7f6d480b22a0 2023-10-07T07:58:21.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.819+0000 7f6d4dadd700 1 --2- 172.21.15.7:0/612991195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d480b22a0 0x7f6d480afa80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d480cf0c0 tx=0x7f6d38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ab9a02db6abbe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:21.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d4d2dc700 1 -- 172.21.15.7:0/612991195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800b820 con 0x7f6d480b22a0 2023-10-07T07:58:21.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d4d2dc700 1 -- 172.21.15.7:0/612991195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3800b980 con 0x7f6d480b22a0 2023-10-07T07:58:21.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d4d2dc700 1 -- 172.21.15.7:0/612991195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800bd20 con 0x7f6d480b22a0 2023-10-07T07:58:21.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d50542700 1 -- 172.21.15.7:0/612991195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d480b22a0 msgr2=0x7f6d480afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d50542700 1 --2- 172.21.15.7:0/612991195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d480b22a0 0x7f6d480afa80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d480cf0c0 tx=0x7f6d38009250 comp rx=0 tx=0).stop 2023-10-07T07:58:21.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d50542700 1 -- 172.21.15.7:0/612991195 shutdown_connections 2023-10-07T07:58:21.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d50542700 1 --2- 172.21.15.7:0/612991195 >> v2:172.21.15.172:3300/0 conn(0x7f6d48156f30 0x7f6d480b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d50542700 1 --2- 172.21.15.7:0/612991195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d480b22a0 0x7f6d480afa80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d50542700 1 -- 172.21.15.7:0/612991195 >> 172.21.15.7:0/612991195 conn(0x7f6d4806d350 msgr2=0x7f6d4806d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:21.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d50542700 1 -- 172.21.15.7:0/612991195 shutdown_connections 2023-10-07T07:58:21.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.820+0000 7f6d50542700 1 -- 172.21.15.7:0/612991195 wait complete. 2023-10-07T07:58:21.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.821+0000 7f6d50542700 1 Processor -- start 2023-10-07T07:58:21.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.822+0000 7f6d50542700 1 -- start start 2023-10-07T07:58:21.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.822+0000 7f6d50542700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d480b22a0 0x7f6d480c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.822+0000 7f6d50542700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d48156f30 0x7f6d4814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.822+0000 7f6d50542700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d48157300 0x7f6d480ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.822+0000 7f6d50542700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4814ee40 con 0x7f6d48156f30 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.822+0000 7f6d50542700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4814f070 con 0x7f6d48157300 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.822+0000 7f6d50542700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4814f350 con 0x7f6d480b22a0 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.822+0000 7f6d4dadd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d48156f30 0x7f6d4814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4dadd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d48156f30 0x7f6d4814e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46844/0 (socket says 172.21.15.7:46844) 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4dadd700 1 -- 172.21.15.7:0/3484650498 learned_addr learned my addr 172.21.15.7:0/3484650498 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4e2de700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d480b22a0 0x7f6d480c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4eadf700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d48157300 0x7f6d480ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4dadd700 1 -- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d480b22a0 msgr2=0x7f6d480c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4dadd700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d480b22a0 0x7f6d480c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4dadd700 1 -- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d48157300 msgr2=0x7f6d480ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:21.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4dadd700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d48157300 0x7f6d480ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4dadd700 1 -- 172.21.15.7:0/3484650498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380094f0 con 0x7f6d48156f30 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d4dadd700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d48156f30 0x7f6d4814e900 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d380030c0 tx=0x7f6d38016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/3484650498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38011e60 con 0x7f6d48156f30 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/3484650498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d38020070 con 0x7f6d48156f30 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4814f630 con 0x7f6d48156f30 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/3484650498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3801b6a0 con 0x7f6d48156f30 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.823+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4814fbf0 con 0x7f6d48156f30 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.824+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4814fde0 con 0x7f6d48156f30 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.825+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/3484650498 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d38022030 con 0x7f6d48156f30 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.825+0000 7f6d3e7fc700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6d2c066f30 0x7f6d2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.825+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/3484650498 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6d380864b0 con 0x7f6d48156f30 2023-10-07T07:58:21.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.825+0000 7f6d4e2de700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6d2c066f30 0x7f6d2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:21.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.826+0000 7f6d4e2de700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6d2c066f30 0x7f6d2c0693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400ab30 tx=0x7f6d44008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:21.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:21.830+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/3484650498 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d38053720 con 0x7f6d48156f30 2023-10-07T07:58:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.031+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d4814fde0 con 0x7f6d48156f30 2023-10-07T07:58:22.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.031+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/3484650498 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d38056d40 con 0x7f6d48156f30 2023-10-07T07:58:22.033 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-07T07:58:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6d2c066f30 msgr2=0x7f6d2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6d2c066f30 0x7f6d2c0693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400ab30 tx=0x7f6d44008040 comp rx=0 tx=0).stop 2023-10-07T07:58:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d48156f30 msgr2=0x7f6d4814e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d48156f30 0x7f6d4814e900 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d380030c0 tx=0x7f6d38016000 comp rx=0 tx=0).stop 2023-10-07T07:58:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 shutdown_connections 2023-10-07T07:58:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6d2c066f30 0x7f6d2c0693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d48157300 0x7f6d480ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d480b22a0 0x7f6d480c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 --2- 172.21.15.7:0/3484650498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d48156f30 0x7f6d4814e900 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 >> 172.21.15.7:0/3484650498 conn(0x7f6d4806d350 msgr2=0x7f6d4815ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 shutdown_connections 2023-10-07T07:58:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.034+0000 7f6d50542700 1 -- 172.21.15.7:0/3484650498 wait complete. 2023-10-07T07:58:22.058 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) Mostly readdir acquires caps faster than the mds recalls, so the cap 2023-10-07T07:58:22.058 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T07:58:22.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-10-07T07:58:22.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.189+0000 7fe91c2e4700 1 Processor -- start 2023-10-07T07:58:22.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.189+0000 7fe91c2e4700 1 -- start start 2023-10-07T07:58:22.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.190+0000 7fe91c2e4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9140c9af0 0x7fe91415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.190+0000 7fe91c2e4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 0x7fe91406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.190+0000 7fe91c2e4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9140c9ec0 0x7fe91406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.190+0000 7fe91c2e4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9140b03a0 con 0x7fe9140c9ec0 2023-10-07T07:58:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.190+0000 7fe91c2e4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91406e590 con 0x7fe9140c9af0 2023-10-07T07:58:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.190+0000 7fe91c2e4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91415a330 con 0x7fe914063230 2023-10-07T07:58:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a881700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9140c9ec0 0x7fe91406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a881700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9140c9ec0 0x7fe91406e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46854/0 (socket says 172.21.15.7:46854) 2023-10-07T07:58:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a881700 1 -- 172.21.15.7:0/915997292 learned_addr learned my addr 172.21.15.7:0/915997292 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a080700 1 --2- 172.21.15.7:0/915997292 >> v2:172.21.15.172:3300/0 conn(0x7fe9140c9af0 0x7fe91415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91987f700 1 --2- 172.21.15.7:0/915997292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 0x7fe91406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a881700 1 -- 172.21.15.7:0/915997292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 msgr2=0x7fe91406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a881700 1 --2- 172.21.15.7:0/915997292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 0x7fe91406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a881700 1 -- 172.21.15.7:0/915997292 >> v2:172.21.15.172:3300/0 conn(0x7fe9140c9af0 msgr2=0x7fe91415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a881700 1 --2- 172.21.15.7:0/915997292 >> v2:172.21.15.172:3300/0 conn(0x7fe9140c9af0 0x7fe91415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a881700 1 -- 172.21.15.7:0/915997292 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91415a610 con 0x7fe9140c9ec0 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91987f700 1 --2- 172.21.15.7:0/915997292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 0x7fe91406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91a881700 1 --2- 172.21.15.7:0/915997292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 0x7fe91406e050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9140ca400 tx=0x7fe90c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9cc8464bcf095d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91907e700 1 -- 172.21.15.7:0/915997292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00e070 con 0x7fe9140c9ec0 2023-10-07T07:58:22.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91907e700 1 -- 172.21.15.7:0/915997292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c00ba20 con 0x7fe9140c9ec0 2023-10-07T07:58:22.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.191+0000 7fe91907e700 1 -- 172.21.15.7:0/915997292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0092e0 con 0x7fe9140c9ec0 2023-10-07T07:58:22.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.192+0000 7fe91c2e4700 1 -- 172.21.15.7:0/915997292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 msgr2=0x7fe91406e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:22.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.192+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/915997292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 0x7fe91406e050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9140ca400 tx=0x7fe90c009840 comp rx=0 tx=0).stop 2023-10-07T07:58:22.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.192+0000 7fe91c2e4700 1 -- 172.21.15.7:0/915997292 shutdown_connections 2023-10-07T07:58:22.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.192+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/915997292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 0x7fe91406e050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.192+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/915997292 >> v2:172.21.15.172:3300/0 conn(0x7fe9140c9af0 0x7fe91415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.192+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/915997292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 0x7fe91406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.192+0000 7fe91c2e4700 1 -- 172.21.15.7:0/915997292 >> 172.21.15.7:0/915997292 conn(0x7fe9140cd1a0 msgr2=0x7fe9140c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.192+0000 7fe91c2e4700 1 -- 172.21.15.7:0/915997292 shutdown_connections 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.192+0000 7fe91c2e4700 1 -- 172.21.15.7:0/915997292 wait complete. 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.193+0000 7fe91c2e4700 1 Processor -- start 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.193+0000 7fe91c2e4700 1 -- start start 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.194+0000 7fe91c2e4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 0x7fe914147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.194+0000 7fe91c2e4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9140c9af0 0x7fe9141694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.194+0000 7fe91c2e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 0x7fe914157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.194+0000 7fe91c2e4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914156f70 con 0x7fe9140c9ec0 2023-10-07T07:58:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.194+0000 7fe91c2e4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91415b3d0 con 0x7fe9140c9af0 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.194+0000 7fe91c2e4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140c8620 con 0x7fe914063230 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.194+0000 7fe91a881700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 0x7fe914157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.194+0000 7fe91a881700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 0x7fe914157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46870/0 (socket says 172.21.15.7:46870) 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.194+0000 7fe91a881700 1 -- 172.21.15.7:0/3044715211 learned_addr learned my addr 172.21.15.7:0/3044715211 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91987f700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9140c9af0 0x7fe9141694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91a080700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 0x7fe914147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91a881700 1 -- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 msgr2=0x7fe914147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91a881700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 0x7fe914147160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91a881700 1 -- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9140c9af0 msgr2=0x7fe9141694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91a881700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9140c9af0 0x7fe9141694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:22.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91a881700 1 -- 172.21.15.7:0/3044715211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c009ae0 con 0x7fe9140c9ec0 2023-10-07T07:58:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91987f700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9140c9af0 0x7fe9141694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91a881700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 0x7fe914157400 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe91406d7c0 tx=0x7fe90c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe9067fc700 1 -- 172.21.15.7:0/3044715211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0032f0 con 0x7fe9140c9ec0 2023-10-07T07:58:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe9067fc700 1 -- 172.21.15.7:0/3044715211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c026460 con 0x7fe9140c9ec0 2023-10-07T07:58:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe9067fc700 1 -- 172.21.15.7:0/3044715211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c026800 con 0x7fe9140c9ec0 2023-10-07T07:58:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9141476a0 con 0x7fe9140c9ec0 2023-10-07T07:58:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.195+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9140c8240 con 0x7fe9140c9ec0 2023-10-07T07:58:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.196+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9140c8460 con 0x7fe9140c9ec0 2023-10-07T07:58:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.196+0000 7fe9067fc700 1 -- 172.21.15.7:0/3044715211 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe90c003040 con 0x7fe9140c9ec0 2023-10-07T07:58:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.197+0000 7fe9067fc700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8f8066f30 0x7fe8f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.197+0000 7fe9067fc700 1 -- 172.21.15.7:0/3044715211 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe90c086550 con 0x7fe9140c9ec0 2023-10-07T07:58:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.197+0000 7fe91a080700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8f8066f30 0x7fe8f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.198+0000 7fe91a080700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8f8066f30 0x7fe8f80693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe910009db0 tx=0x7fe9100091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.201+0000 7fe9067fc700 1 -- 172.21.15.7:0/3044715211 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe90c0537c0 con 0x7fe9140c9ec0 2023-10-07T07:58:22.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:22.486+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7fe9140716c0 con 0x7fe9140c9ec0 2023-10-07T07:58:23.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.341+0000 7fe9067fc700 1 -- 172.21.15.7:0/3044715211 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v47) v1 ==== 168+0+0 (secure 0 0 0) 0x7fe90c056de0 con 0x7fe9140c9ec0 2023-10-07T07:58:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8f8066f30 msgr2=0x7fe8f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8f8066f30 0x7fe8f80693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe910009db0 tx=0x7fe9100091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 msgr2=0x7fe914157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 0x7fe914157400 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe91406d7c0 tx=0x7fe90c018000 comp rx=0 tx=0).stop 2023-10-07T07:58:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 shutdown_connections 2023-10-07T07:58:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8f8066f30 0x7fe8f80693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9140c9ec0 0x7fe914157400 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe914063230 0x7fe914147160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 --2- 172.21.15.7:0/3044715211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9140c9af0 0x7fe9141694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 >> 172.21.15.7:0/3044715211 conn(0x7fe9140cd1a0 msgr2=0x7fe914066ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 shutdown_connections 2023-10-07T07:58:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.344+0000 7fe91c2e4700 1 -- 172.21.15.7:0/3044715211 wait complete. 2023-10-07T07:58:23.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-07T07:58:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.501+0000 7f043264f700 1 Processor -- start 2023-10-07T07:58:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.501+0000 7f043264f700 1 -- start start 2023-10-07T07:58:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.502+0000 7f043264f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c1576b0 0x7f042c06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.502+0000 7f043264f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f042c06a650 0x7f042c0c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.502+0000 7f043264f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042c0cc0c0 con 0x7f042c068000 2023-10-07T07:58:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.502+0000 7f043264f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042c0cb4b0 con 0x7f042c06a650 2023-10-07T07:58:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.502+0000 7f043264f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c0cb790 con 0x7f042c1576b0 2023-10-07T07:58:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.502+0000 7f042bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f042c068000 0x7f042c0683d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46900/0 (socket says 172.21.15.7:46900) 2023-10-07T07:58:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.502+0000 7f042bfff700 1 -- 172.21.15.7:0/1791608021 learned_addr learned my addr 172.21.15.7:0/1791608021 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.502+0000 7f042b7fe700 1 --2- 172.21.15.7:0/1791608021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c1576b0 0x7f042c06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.502+0000 7f0430bec700 1 --2- 172.21.15.7:0/1791608021 >> v2:172.21.15.172:3300/0 conn(0x7f042c06a650 0x7f042c0c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042affd700 1 -- 172.21.15.7:0/1791608021 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709677565 0 0) 0x7f042c0cc0c0 con 0x7f042c068000 2023-10-07T07:58:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042affd700 1 -- 172.21.15.7:0/1791608021 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0414003570 con 0x7f042c068000 2023-10-07T07:58:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042b7fe700 1 -- 172.21.15.7:0/1791608021 >> v2:172.21.15.172:3300/0 conn(0x7f042c06a650 msgr2=0x7f042c0c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042b7fe700 1 --2- 172.21.15.7:0/1791608021 >> v2:172.21.15.172:3300/0 conn(0x7f042c06a650 0x7f042c0c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042b7fe700 1 -- 172.21.15.7:0/1791608021 >> v1:172.21.15.7:6789/0 conn(0x7f042c068000 legacy=0x7f042c0683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042b7fe700 1 -- 172.21.15.7:0/1791608021 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c0cba70 con 0x7f042c1576b0 2023-10-07T07:58:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f0430bec700 1 --2- 172.21.15.7:0/1791608021 >> v2:172.21.15.172:3300/0 conn(0x7f042c06a650 0x7f042c0c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042b7fe700 1 --2- 172.21.15.7:0/1791608021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c1576b0 0x7f042c06bff0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f04200098e0 tx=0x7f04200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e14dd535f4a1525 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042affd700 1 -- 172.21.15.7:0/1791608021 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420011630 con 0x7f042c1576b0 2023-10-07T07:58:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042affd700 1 -- 172.21.15.7:0/1791608021 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0420011790 con 0x7f042c1576b0 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.503+0000 7f042affd700 1 -- 172.21.15.7:0/1791608021 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420011b30 con 0x7f042c1576b0 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.504+0000 7f043264f700 1 -- 172.21.15.7:0/1791608021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c1576b0 msgr2=0x7f042c06bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.504+0000 7f043264f700 1 --2- 172.21.15.7:0/1791608021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c1576b0 0x7f042c06bff0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f04200098e0 tx=0x7f04200091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.504+0000 7f043264f700 1 -- 172.21.15.7:0/1791608021 shutdown_connections 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.504+0000 7f043264f700 1 --2- 172.21.15.7:0/1791608021 >> v2:172.21.15.172:3300/0 conn(0x7f042c06a650 0x7f042c0c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.504+0000 7f043264f700 1 --2- 172.21.15.7:0/1791608021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c1576b0 0x7f042c06bff0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.504+0000 7f043264f700 1 -- 172.21.15.7:0/1791608021 >> 172.21.15.7:0/1791608021 conn(0x7f042c0c9c30 msgr2=0x7f042c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.504+0000 7f043264f700 1 -- 172.21.15.7:0/1791608021 shutdown_connections 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.504+0000 7f043264f700 1 -- 172.21.15.7:0/1791608021 wait complete. 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.505+0000 7f043264f700 1 Processor -- start 2023-10-07T07:58:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.505+0000 7f043264f700 1 -- start start 2023-10-07T07:58:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.506+0000 7f043264f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c068000 0x7f042c145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.506+0000 7f043264f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042c06a650 0x7f042c158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.506+0000 7f043264f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1576b0 0x7f042c155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.506+0000 7f043264f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c1459a0 con 0x7f042c06a650 2023-10-07T07:58:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.506+0000 7f043264f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c159870 con 0x7f042c1576b0 2023-10-07T07:58:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.506+0000 7f043264f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c159ac0 con 0x7f042c068000 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.506+0000 7f042b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042c06a650 0x7f042c158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.506+0000 7f042b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042c06a650 0x7f042c158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46880/0 (socket says 172.21.15.7:46880) 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.506+0000 7f042b7fe700 1 -- 172.21.15.7:0/3577006835 learned_addr learned my addr 172.21.15.7:0/3577006835 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f042bfff700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c068000 0x7f042c145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f0430bec700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1576b0 0x7f042c155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f042b7fe700 1 -- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c068000 msgr2=0x7f042c145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f042b7fe700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c068000 0x7f042c145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f042b7fe700 1 -- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1576b0 msgr2=0x7f042c155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f042b7fe700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1576b0 0x7f042c155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f042b7fe700 1 -- 172.21.15.7:0/3577006835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0424009a20 con 0x7f042c06a650 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f0430bec700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1576b0 0x7f042c155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f042bfff700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c068000 0x7f042c145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f042b7fe700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042c06a650 0x7f042c158490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0420013000 tx=0x7f0420017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f0413fff700 1 -- 172.21.15.7:0/3577006835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420011900 con 0x7f042c06a650 2023-10-07T07:58:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f0413fff700 1 -- 172.21.15.7:0/3577006835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0420011a80 con 0x7f042c06a650 2023-10-07T07:58:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f0413fff700 1 -- 172.21.15.7:0/3577006835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420025670 con 0x7f042c06a650 2023-10-07T07:58:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0420009420 con 0x7f042c06a650 2023-10-07T07:58:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.507+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c15a080 con 0x7f042c06a650 2023-10-07T07:58:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.508+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c04c3c0 con 0x7f042c06a650 2023-10-07T07:58:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.509+0000 7f0413fff700 1 -- 172.21.15.7:0/3577006835 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0420023070 con 0x7f042c06a650 2023-10-07T07:58:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.509+0000 7f0413fff700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0418066f30 0x7f04180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.509+0000 7f0413fff700 1 -- 172.21.15.7:0/3577006835 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0420085960 con 0x7f042c06a650 2023-10-07T07:58:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.509+0000 7f042bfff700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0418066f30 0x7f04180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.510+0000 7f042bfff700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0418066f30 0x7f04180693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f041c001360 tx=0x7f041c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:23.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.514+0000 7f0413fff700 1 -- 172.21.15.7:0/3577006835 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0420052bd0 con 0x7f042c06a650 2023-10-07T07:58:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.719+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f042c0cbd50 con 0x7f042c06a650 2023-10-07T07:58:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.723+0000 7f0413fff700 1 -- 172.21.15.7:0/3577006835 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f042002a0c0 con 0x7f042c06a650 2023-10-07T07:58:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0418066f30 msgr2=0x7f04180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0418066f30 0x7f04180693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f041c001360 tx=0x7f041c009040 comp rx=0 tx=0).stop 2023-10-07T07:58:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042c06a650 msgr2=0x7f042c158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042c06a650 0x7f042c158490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0420013000 tx=0x7f0420017000 comp rx=0 tx=0).stop 2023-10-07T07:58:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 shutdown_connections 2023-10-07T07:58:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0418066f30 0x7f04180693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1576b0 0x7f042c155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f042c068000 0x7f042c145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 --2- 172.21.15.7:0/3577006835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042c06a650 0x7f042c158490 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 >> 172.21.15.7:0/3577006835 conn(0x7f042c0c9c30 msgr2=0x7f042c1565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 shutdown_connections 2023-10-07T07:58:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:23.725+0000 7f043264f700 1 -- 172.21.15.7:0/3577006835 wait complete. 2023-10-07T07:58:23.740 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-07T07:58:23.740 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T07:58:23.783 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T07:58:23.834 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-07T07:58:23.834 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-07T07:58:23.834 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T07:58:23.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:23.878 INFO:teuthology.orchestra.run:waiting for 900 2023-10-07T07:58:23.878 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:58:23.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:23.879 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-07T07:58:23.964 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi172.front.sepia.ceph.com... 2023-10-07T07:58:23.964 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T07:58:23.964 DEBUG:teuthology.orchestra.run.smithi172:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:24.011 INFO:teuthology.orchestra.run:waiting for 900 2023-10-07T07:58:24.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:58:24.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:24.012 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-07T07:58:24.082 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-07T07:58:24.082 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T07:58:24.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-07T07:58:24.127 INFO:teuthology.orchestra.run:waiting for 900 2023-10-07T07:58:24.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:58:24.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:24.127 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-07T07:58:24.191 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi172.front.sepia.ceph.com... 2023-10-07T07:58:24.191 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T07:58:24.191 DEBUG:teuthology.orchestra.run.smithi172:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-07T07:58:24.227 INFO:teuthology.orchestra.run:waiting for 900 2023-10-07T07:58:24.227 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:58:24.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:24.228 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-07T07:58:24.288 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-07T07:58:24.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:58:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.428+0000 7f4fb37fc700 1 Processor -- start 2023-10-07T07:58:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.428+0000 7f4fb37fc700 1 -- start start 2023-10-07T07:58:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.429+0000 7f4fb37fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac146f60 0x7f4fac06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.429+0000 7f4fb37fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4fac147330 0x7f4fac0c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.429+0000 7f4fb37fc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac06ae70 con 0x7f4fac06a930 2023-10-07T07:58:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.429+0000 7f4fb37fc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac06b150 con 0x7f4fac147330 2023-10-07T07:58:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.429+0000 7f4fb37fc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac06b430 con 0x7f4fac146f60 2023-10-07T07:58:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.429+0000 7f4fb1d99700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4fac147330 0x7f4fac0c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.429+0000 7f4fb0d97700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4fac06a930 0x7f4fac0cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46916/0 (socket says 172.21.15.7:46916) 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.429+0000 7f4fb0d97700 1 -- 172.21.15.7:0/2185592790 learned_addr learned my addr 172.21.15.7:0/2185592790 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.429+0000 7f4fb1598700 1 --2- 172.21.15.7:0/2185592790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac146f60 0x7f4fac06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fb1d99700 1 -- 172.21.15.7:0/2185592790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac146f60 msgr2=0x7f4fac06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fb1d99700 1 --2- 172.21.15.7:0/2185592790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac146f60 0x7f4fac06a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fb1d99700 1 -- 172.21.15.7:0/2185592790 >> v1:172.21.15.7:6789/0 conn(0x7f4fac06a930 legacy=0x7f4fac0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fb1d99700 1 -- 172.21.15.7:0/2185592790 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac06b710 con 0x7f4fac147330 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fb1598700 1 --2- 172.21.15.7:0/2185592790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac146f60 0x7f4fac06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fb1d99700 1 --2- 172.21.15.7:0/2185592790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac147330 0x7f4fac0c79d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009e60 tx=0x7f4fa4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb7add2eb0df659 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fa3fff700 1 -- 172.21.15.7:0/2185592790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400e070 con 0x7f4fac147330 2023-10-07T07:58:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fa3fff700 1 -- 172.21.15.7:0/2185592790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa400bc30 con 0x7f4fac147330 2023-10-07T07:58:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fa3fff700 1 -- 172.21.15.7:0/2185592790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4012430 con 0x7f4fac147330 2023-10-07T07:58:24.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.430+0000 7f4fb37fc700 1 -- 172.21.15.7:0/2185592790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac147330 msgr2=0x7f4fac0c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.431+0000 7f4fb37fc700 1 --2- 172.21.15.7:0/2185592790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac147330 0x7f4fac0c79d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009e60 tx=0x7f4fa4009780 comp rx=0 tx=0).stop 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.431+0000 7f4fb37fc700 1 -- 172.21.15.7:0/2185592790 shutdown_connections 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.431+0000 7f4fb37fc700 1 --2- 172.21.15.7:0/2185592790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac147330 0x7f4fac0c79d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.431+0000 7f4fb37fc700 1 --2- 172.21.15.7:0/2185592790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac146f60 0x7f4fac06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.431+0000 7f4fb37fc700 1 -- 172.21.15.7:0/2185592790 >> 172.21.15.7:0/2185592790 conn(0x7f4fac068bd0 msgr2=0x7f4fac0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.431+0000 7f4fb37fc700 1 -- 172.21.15.7:0/2185592790 shutdown_connections 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.431+0000 7f4fb37fc700 1 -- 172.21.15.7:0/2185592790 wait complete. 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.432+0000 7f4fb37fc700 1 Processor -- start 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.432+0000 7f4fb37fc700 1 -- start start 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb37fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac06a930 0x7f4fac15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb37fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fac146f60 0x7f4fac161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb37fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac147330 0x7f4fac0c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb37fc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0c9120 con 0x7f4fac06a930 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb37fc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac16f0c0 con 0x7f4fac146f60 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb37fc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac16e460 con 0x7f4fac147330 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb1598700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac06a930 0x7f4fac15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb1598700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac06a930 0x7f4fac15d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46902/0 (socket says 172.21.15.7:46902) 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb1598700 1 -- 172.21.15.7:0/3787905513 learned_addr learned my addr 172.21.15.7:0/3787905513 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb0d97700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fac146f60 0x7f4fac161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb1d99700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac147330 0x7f4fac0c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb1598700 1 -- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac147330 msgr2=0x7f4fac0c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb1598700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac147330 0x7f4fac0c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb1598700 1 -- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fac146f60 msgr2=0x7f4fac161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb1598700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fac146f60 0x7f4fac161c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.433+0000 7f4fb1598700 1 -- 172.21.15.7:0/3787905513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4009a20 con 0x7f4fac06a930 2023-10-07T07:58:24.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.434+0000 7f4fb1598700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac06a930 0x7f4fac15d280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800e8d0 tx=0x7f4fa8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:24.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.434+0000 7f4fa17fa700 1 -- 172.21.15.7:0/3787905513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa800ee70 con 0x7f4fac06a930 2023-10-07T07:58:24.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.434+0000 7f4fa17fa700 1 -- 172.21.15.7:0/3787905513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa800c5e0 con 0x7f4fac06a930 2023-10-07T07:58:24.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.434+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac16e790 con 0x7f4fac06a930 2023-10-07T07:58:24.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.434+0000 7f4fa17fa700 1 -- 172.21.15.7:0/3787905513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa800c980 con 0x7f4fac06a930 2023-10-07T07:58:24.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.434+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac16ed50 con 0x7f4fac06a930 2023-10-07T07:58:24.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.435+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac067dd0 con 0x7f4fac06a930 2023-10-07T07:58:24.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.435+0000 7f4fa17fa700 1 -- 172.21.15.7:0/3787905513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4fa8013410 con 0x7f4fac06a930 2023-10-07T07:58:24.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.436+0000 7f4fa17fa700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4f90066f30 0x7f4f900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.436+0000 7f4fa17fa700 1 -- 172.21.15.7:0/3787905513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4fa8084390 con 0x7f4fac06a930 2023-10-07T07:58:24.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.436+0000 7f4fb0d97700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4f90066f30 0x7f4f900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.436+0000 7f4fb0d97700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4f90066f30 0x7f4f900693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0afb10 tx=0x7f4f9c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:24.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.440+0000 7f4fa17fa700 1 -- 172.21.15.7:0/3787905513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fa8051600 con 0x7f4fac06a930 2023-10-07T07:58:24.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.689+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fac16f3a0 con 0x7f4fac06a930 2023-10-07T07:58:24.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.689+0000 7f4fa17fa700 1 -- 172.21.15.7:0/3787905513 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f4fa8054c20 con 0x7f4fac06a930 2023-10-07T07:58:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:24.691 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2318752844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2318752844},{"type":"v1","addr":"172.21.15.172:6836","nonce":2318752844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1231430544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1231430544},{"type":"v1","addr":"172.21.15.172:6837","nonce":1231430544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1421504798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1421504798},{"type":"v1","addr":"172.21.15.7:6833","nonce":1421504798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:57:50.139330+0000","modified":"2023-10-07T07:57:53.727131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T07:58:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4f90066f30 msgr2=0x7f4f900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4f90066f30 0x7f4f900693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0afb10 tx=0x7f4f9c009040 comp rx=0 tx=0).stop 2023-10-07T07:58:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac06a930 msgr2=0x7f4fac15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac06a930 0x7f4fac15d280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800e8d0 tx=0x7f4fa8008040 comp rx=0 tx=0).stop 2023-10-07T07:58:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 shutdown_connections 2023-10-07T07:58:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4f90066f30 0x7f4f900693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fac147330 0x7f4fac0c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac06a930 0x7f4fac15d280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 --2- 172.21.15.7:0/3787905513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fac146f60 0x7f4fac161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 >> 172.21.15.7:0/3787905513 conn(0x7f4fac068bd0 msgr2=0x7f4fac06ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 shutdown_connections 2023-10-07T07:58:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.692+0000 7f4fb37fc700 1 -- 172.21.15.7:0/3787905513 wait complete. 2023-10-07T07:58:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2023-10-07T07:58:24.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.837+0000 7f203825d700 1 Processor -- start 2023-10-07T07:58:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.837+0000 7f203825d700 1 -- start start 2023-10-07T07:58:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f203825d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20300cfe00 0x7f20300cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f203825d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2030069760 0x7f203015d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f203825d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2030159fd0 con 0x7f2030069390 2023-10-07T07:58:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f203825d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203015ab70 con 0x7f2030069760 2023-10-07T07:58:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f203825d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20300ca960 con 0x7f20300cfe00 2023-10-07T07:58:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f20357f8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20300cfe00 0x7f20300cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f20367fa700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2030069760 0x7f203015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f20357f8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20300cfe00 0x7f20300cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:48982/0 (socket says 172.21.15.7:48982) 2023-10-07T07:58:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f20357f8700 1 -- 172.21.15.7:0/596833257 learned_addr learned my addr 172.21.15.7:0/596833257 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.838+0000 7f2035ff9700 1 --1- 172.21.15.7:0/596833257 >> v1:172.21.15.172:6789/0 conn(0x7f2030069390 0x7f203015a460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:51200/0 (socket says 172.21.15.7:51200) 2023-10-07T07:58:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.839+0000 7f20357f8700 1 -- 172.21.15.7:0/596833257 >> v2:172.21.15.172:3300/0 conn(0x7f2030069760 msgr2=0x7f203015d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.839+0000 7f20357f8700 1 --2- 172.21.15.7:0/596833257 >> v2:172.21.15.172:3300/0 conn(0x7f2030069760 0x7f203015d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.839+0000 7f20357f8700 1 -- 172.21.15.7:0/596833257 >> v1:172.21.15.172:6789/0 conn(0x7f2030069390 legacy=0x7f203015a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.839+0000 7f20357f8700 1 -- 172.21.15.7:0/596833257 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20300cac40 con 0x7f20300cfe00 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.839+0000 7f20367fa700 1 --2- 172.21.15.7:0/596833257 >> v2:172.21.15.172:3300/0 conn(0x7f2030069760 0x7f203015d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.839+0000 7f20357f8700 1 --2- 172.21.15.7:0/596833257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20300cfe00 0x7f20300cb2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f20200099b0 tx=0x7f2020009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c84ac0fd26609e12 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.839+0000 7f2034ff7700 1 -- 172.21.15.7:0/596833257 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020009ea0 con 0x7f20300cfe00 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.839+0000 7f2034ff7700 1 -- 172.21.15.7:0/596833257 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000b860 con 0x7f20300cfe00 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.839+0000 7f2034ff7700 1 -- 172.21.15.7:0/596833257 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000bc00 con 0x7f20300cfe00 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.840+0000 7f203825d700 1 -- 172.21.15.7:0/596833257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20300cfe00 msgr2=0x7f20300cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.840+0000 7f203825d700 1 --2- 172.21.15.7:0/596833257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20300cfe00 0x7f20300cb2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f20200099b0 tx=0x7f2020009250 comp rx=0 tx=0).stop 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.840+0000 7f203825d700 1 -- 172.21.15.7:0/596833257 shutdown_connections 2023-10-07T07:58:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.840+0000 7f203825d700 1 --2- 172.21.15.7:0/596833257 >> v2:172.21.15.172:3300/0 conn(0x7f2030069760 0x7f203015d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.840+0000 7f203825d700 1 --2- 172.21.15.7:0/596833257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20300cfe00 0x7f20300cb2b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.840+0000 7f203825d700 1 -- 172.21.15.7:0/596833257 >> 172.21.15.7:0/596833257 conn(0x7f20300d0280 msgr2=0x7f20300d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:24.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.840+0000 7f203825d700 1 -- 172.21.15.7:0/596833257 shutdown_connections 2023-10-07T07:58:24.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.840+0000 7f203825d700 1 -- 172.21.15.7:0/596833257 wait complete. 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.841+0000 7f203825d700 1 Processor -- start 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.841+0000 7f203825d700 1 -- start start 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f203825d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2030069390 0x7f20300c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f203825d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030069760 0x7f2030147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f203825d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20300cfe00 0x7f20301555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f203825d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20300c99f0 con 0x7f2030069390 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f203825d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20300b3440 con 0x7f20300cfe00 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f203825d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20300c9c90 con 0x7f2030069760 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f2035ff9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2030069390 0x7f20300c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f2035ff9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2030069390 0x7f20300c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46912/0 (socket says 172.21.15.7:46912) 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f2035ff9700 1 -- 172.21.15.7:0/1924452114 learned_addr learned my addr 172.21.15.7:0/1924452114 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f20357f8700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030069760 0x7f2030147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f20367fa700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20300cfe00 0x7f20301555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f2035ff9700 1 -- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030069760 msgr2=0x7f2030147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f2035ff9700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030069760 0x7f2030147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f2035ff9700 1 -- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20300cfe00 msgr2=0x7f20301555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f2035ff9700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20300cfe00 0x7f20301555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.842+0000 7f2035ff9700 1 -- 172.21.15.7:0/1924452114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028009a20 con 0x7f2030069390 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.843+0000 7f2035ff9700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2030069390 0x7f20300c7330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c560 tx=0x7f202c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.843+0000 7f2025ffb700 1 -- 172.21.15.7:0/1924452114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c005a70 con 0x7f2030069390 2023-10-07T07:58:24.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.843+0000 7f2025ffb700 1 -- 172.21.15.7:0/1924452114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c005bd0 con 0x7f2030069390 2023-10-07T07:58:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.843+0000 7f2025ffb700 1 -- 172.21.15.7:0/1924452114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c0070d0 con 0x7f2030069390 2023-10-07T07:58:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.843+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20200094f0 con 0x7f2030069390 2023-10-07T07:58:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.843+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20300c7870 con 0x7f2030069390 2023-10-07T07:58:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.844+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20300b3660 con 0x7f2030069390 2023-10-07T07:58:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.845+0000 7f2025ffb700 1 -- 172.21.15.7:0/1924452114 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f202c007440 con 0x7f2030069390 2023-10-07T07:58:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.845+0000 7f2025ffb700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2014066f30 0x7f20140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.845+0000 7f2025ffb700 1 -- 172.21.15.7:0/1924452114 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f202c083920 con 0x7f2030069390 2023-10-07T07:58:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.845+0000 7f20357f8700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2014066f30 0x7f20140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.846+0000 7f20357f8700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2014066f30 0x7f20140693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f20200099b0 tx=0x7f2020017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:24.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:24.850+0000 7f2025ffb700 1 -- 172.21.15.7:0/1924452114 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20300b3660 con 0x7f2030069390 2023-10-07T07:58:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.052+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20300ca1b0 con 0x7f2030069390 2023-10-07T07:58:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.053+0000 7f2025ffb700 1 -- 172.21.15.7:0/1924452114 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f202c017020 con 0x7f2030069390 2023-10-07T07:58:25.055 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:25.055 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2014066f30 msgr2=0x7f20140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2014066f30 0x7f20140693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f20200099b0 tx=0x7f2020017000 comp rx=0 tx=0).stop 2023-10-07T07:58:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2030069390 msgr2=0x7f20300c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2030069390 0x7f20300c7330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c560 tx=0x7f202c007f70 comp rx=0 tx=0).stop 2023-10-07T07:58:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 shutdown_connections 2023-10-07T07:58:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2014066f30 0x7f20140693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20300cfe00 0x7f20301555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2030069390 0x7f20300c7330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 --2- 172.21.15.7:0/1924452114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2030069760 0x7f2030147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 >> 172.21.15.7:0/1924452114 conn(0x7f20300d0280 msgr2=0x7f20300c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 shutdown_connections 2023-10-07T07:58:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.056+0000 7f203825d700 1 -- 172.21.15.7:0/1924452114 wait complete. 2023-10-07T07:58:25.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T07:57:48.285077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T07:57:49.111979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T07:58:25.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:58:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.205+0000 7f44111ee700 1 Processor -- start 2023-10-07T07:58:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.205+0000 7f44111ee700 1 -- start start 2023-10-07T07:58:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.206+0000 7f44111ee700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067610 0x7f440c153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.206+0000 7f44111ee700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f440c1539c0 0x7f440c0aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.206+0000 7f44111ee700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c06f320 con 0x7f440c067240 2023-10-07T07:58:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.206+0000 7f44111ee700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c06a500 con 0x7f440c1539c0 2023-10-07T07:58:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.206+0000 7f44111ee700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c153e00 con 0x7f440c067610 2023-10-07T07:58:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.206+0000 7f440b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f440c1539c0 0x7f440c0aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.206+0000 7f440b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f440c1539c0 0x7f440c0aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46916/0 (socket says 172.21.15.7:46916) 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f440b59e700 1 -- 172.21.15.7:0/3062272900 learned_addr learned my addr 172.21.15.7:0/3062272900 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f440a59c700 1 --2- 172.21.15.7:0/3062272900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067610 0x7f440c153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f440b59e700 1 -- 172.21.15.7:0/3062272900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067610 msgr2=0x7f440c153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f440b59e700 1 --2- 172.21.15.7:0/3062272900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067610 0x7f440c153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f440b59e700 1 -- 172.21.15.7:0/3062272900 >> v1:172.21.15.172:6789/0 conn(0x7f440c067240 legacy=0x7f440c15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f440b59e700 1 -- 172.21.15.7:0/3062272900 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c06e7a0 con 0x7f440c1539c0 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f440b59e700 1 --2- 172.21.15.7:0/3062272900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 0x7f440c0aa980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4404009f20 tx=0x7f4404009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd37a8eacd909f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f4409d9b700 1 -- 172.21.15.7:0/3062272900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440400e070 con 0x7f440c1539c0 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f4409d9b700 1 -- 172.21.15.7:0/3062272900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440400b9d0 con 0x7f440c1539c0 2023-10-07T07:58:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.207+0000 7f4409d9b700 1 -- 172.21.15.7:0/3062272900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44040092e0 con 0x7f440c1539c0 2023-10-07T07:58:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.208+0000 7f44111ee700 1 -- 172.21.15.7:0/3062272900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 msgr2=0x7f440c0aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.208+0000 7f44111ee700 1 --2- 172.21.15.7:0/3062272900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 0x7f440c0aa980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4404009f20 tx=0x7f4404009840 comp rx=0 tx=0).stop 2023-10-07T07:58:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.208+0000 7f44111ee700 1 -- 172.21.15.7:0/3062272900 shutdown_connections 2023-10-07T07:58:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.208+0000 7f44111ee700 1 --2- 172.21.15.7:0/3062272900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 0x7f440c0aa980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.208+0000 7f44111ee700 1 --2- 172.21.15.7:0/3062272900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067610 0x7f440c153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.208+0000 7f44111ee700 1 -- 172.21.15.7:0/3062272900 >> 172.21.15.7:0/3062272900 conn(0x7f440c068100 msgr2=0x7f440c068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.208+0000 7f44111ee700 1 -- 172.21.15.7:0/3062272900 shutdown_connections 2023-10-07T07:58:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.208+0000 7f44111ee700 1 -- 172.21.15.7:0/3062272900 wait complete. 2023-10-07T07:58:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.209+0000 7f44111ee700 1 Processor -- start 2023-10-07T07:58:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.209+0000 7f44111ee700 1 -- start start 2023-10-07T07:58:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f44111ee700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067240 0x7f440c140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f44111ee700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f440c067610 0x7f440c152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f44111ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 0x7f440c152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f44111ee700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c16c740 con 0x7f440c1539c0 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f44111ee700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c16bb30 con 0x7f440c067610 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f44111ee700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c16be10 con 0x7f440c067240 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f440b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 0x7f440c152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f440b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 0x7f440c152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46928/0 (socket says 172.21.15.7:46928) 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f440b59e700 1 -- 172.21.15.7:0/1546159960 learned_addr learned my addr 172.21.15.7:0/1546159960 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f440a59c700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f440c067610 0x7f440c152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.210+0000 7f440ad9d700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067240 0x7f440c140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f440b59e700 1 -- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067240 msgr2=0x7f440c140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f440b59e700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067240 0x7f440c140b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f440b59e700 1 -- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f440c067610 msgr2=0x7f440c152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f440b59e700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f440c067610 0x7f440c152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f440b59e700 1 -- 172.21.15.7:0/1546159960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4404009ae0 con 0x7f440c1539c0 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f440a59c700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f440c067610 0x7f440c152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f440b59e700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 0x7f440c152980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f440c0c4000 tx=0x7f4404018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f43f2ffd700 1 -- 172.21.15.7:0/1546159960 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404008c70 con 0x7f440c1539c0 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f43f2ffd700 1 -- 172.21.15.7:0/1546159960 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404008e10 con 0x7f440c1539c0 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f43f2ffd700 1 -- 172.21.15.7:0/1546159960 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440401c660 con 0x7f440c1539c0 2023-10-07T07:58:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c16c0f0 con 0x7f440c1539c0 2023-10-07T07:58:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.211+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c16cbd0 con 0x7f440c1539c0 2023-10-07T07:58:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.212+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c16c8f0 con 0x7f440c1539c0 2023-10-07T07:58:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.213+0000 7f43f2ffd700 1 -- 172.21.15.7:0/1546159960 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4404003040 con 0x7f440c1539c0 2023-10-07T07:58:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.213+0000 7f43f2ffd700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43f8066f30 0x7f43f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.213+0000 7f43f2ffd700 1 -- 172.21.15.7:0/1546159960 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4404085eb0 con 0x7f440c1539c0 2023-10-07T07:58:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.213+0000 7f440ad9d700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43f8066f30 0x7f43f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.214+0000 7f440ad9d700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43f8066f30 0x7f43f80693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f440c0c1f40 tx=0x7f43fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.217+0000 7f43f2ffd700 1 -- 172.21.15.7:0/1546159960 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f440c16c8f0 con 0x7f440c1539c0 2023-10-07T07:58:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.418+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f440c16cd90 con 0x7f440c1539c0 2023-10-07T07:58:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.419+0000 7f43f2ffd700 1 -- 172.21.15.7:0/1546159960 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4404053120 con 0x7f440c1539c0 2023-10-07T07:58:25.420 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43f8066f30 msgr2=0x7f43f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43f8066f30 0x7f43f80693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f440c0c1f40 tx=0x7f43fc009040 comp rx=0 tx=0).stop 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 msgr2=0x7f440c152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 0x7f440c152980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f440c0c4000 tx=0x7f4404018000 comp rx=0 tx=0).stop 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 shutdown_connections 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43f8066f30 0x7f43f80693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f440c1539c0 0x7f440c152980 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f440c067240 0x7f440c140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 --2- 172.21.15.7:0/1546159960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f440c067610 0x7f440c152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 >> 172.21.15.7:0/1546159960 conn(0x7f440c068100 msgr2=0x7f440c15a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 shutdown_connections 2023-10-07T07:58:25.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.421+0000 7f44111ee700 1 -- 172.21.15.7:0/1546159960 wait complete. 2023-10-07T07:58:25.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T07:58:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.570+0000 7f6caa172700 1 Processor -- start 2023-10-07T07:58:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.570+0000 7f6caa172700 1 -- start start 2023-10-07T07:58:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.571+0000 7f6caa172700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b39b0 0x7f6ca4068100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.571+0000 7f6caa172700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ca40b0190 0x7f6ca415d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.571+0000 7f6caa172700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca4068640 con 0x7f6ca40b14a0 2023-10-07T07:58:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.571+0000 7f6caa172700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca40b1130 con 0x7f6ca40b0190 2023-10-07T07:58:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.571+0000 7f6caa172700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca415a8e0 con 0x7f6ca40b39b0 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.571+0000 7f6ca3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ca40b0190 0x7f6ca415d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.571+0000 7f6ca3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ca40b0190 0x7f6ca415d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46944/0 (socket says 172.21.15.7:46944) 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.571+0000 7f6ca3fff700 1 -- 172.21.15.7:0/3441528639 learned_addr learned my addr 172.21.15.7:0/3441528639 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.571+0000 7f6ca37fe700 1 --2- 172.21.15.7:0/3441528639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b39b0 0x7f6ca4068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.572+0000 7f6ca3fff700 1 -- 172.21.15.7:0/3441528639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b39b0 msgr2=0x7f6ca4068100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.572+0000 7f6ca3fff700 1 --2- 172.21.15.7:0/3441528639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b39b0 0x7f6ca4068100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.572+0000 7f6ca3fff700 1 -- 172.21.15.7:0/3441528639 >> v1:172.21.15.172:6789/0 conn(0x7f6ca40b14a0 legacy=0x7f6ca40afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.572+0000 7f6ca3fff700 1 -- 172.21.15.7:0/3441528639 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4159cd0 con 0x7f6ca40b0190 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.572+0000 7f6ca3fff700 1 --2- 172.21.15.7:0/3441528639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b0190 0x7f6ca415d260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009f20 tx=0x7f6c9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1b10b0796096c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.572+0000 7f6ca27fc700 1 -- 172.21.15.7:0/3441528639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c00e070 con 0x7f6ca40b0190 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.572+0000 7f6ca27fc700 1 -- 172.21.15.7:0/3441528639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c00b9d0 con 0x7f6ca40b0190 2023-10-07T07:58:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.572+0000 7f6ca27fc700 1 -- 172.21.15.7:0/3441528639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0092e0 con 0x7f6ca40b0190 2023-10-07T07:58:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.573+0000 7f6caa172700 1 -- 172.21.15.7:0/3441528639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b0190 msgr2=0x7f6ca415d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.573+0000 7f6caa172700 1 --2- 172.21.15.7:0/3441528639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b0190 0x7f6ca415d260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009f20 tx=0x7f6c9c009840 comp rx=0 tx=0).stop 2023-10-07T07:58:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.573+0000 7f6caa172700 1 -- 172.21.15.7:0/3441528639 shutdown_connections 2023-10-07T07:58:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.573+0000 7f6caa172700 1 --2- 172.21.15.7:0/3441528639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b0190 0x7f6ca415d260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.573+0000 7f6caa172700 1 --2- 172.21.15.7:0/3441528639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b39b0 0x7f6ca4068100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.573+0000 7f6caa172700 1 -- 172.21.15.7:0/3441528639 >> 172.21.15.7:0/3441528639 conn(0x7f6ca406d350 msgr2=0x7f6ca406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.573+0000 7f6caa172700 1 -- 172.21.15.7:0/3441528639 shutdown_connections 2023-10-07T07:58:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.573+0000 7f6caa172700 1 -- 172.21.15.7:0/3441528639 wait complete. 2023-10-07T07:58:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.574+0000 7f6caa172700 1 Processor -- start 2023-10-07T07:58:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.574+0000 7f6caa172700 1 -- start start 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6caa172700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b0190 0x7f6ca40c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6caa172700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b14a0 0x7f6ca40c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6caa172700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca40b39b0 0x7f6ca41474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6caa172700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40c8b00 con 0x7f6ca40b14a0 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6caa172700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40c7ef0 con 0x7f6ca40b39b0 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6caa172700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40c81d0 con 0x7f6ca40b0190 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b14a0 0x7f6ca40c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b14a0 0x7f6ca40c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46952/0 (socket says 172.21.15.7:46952) 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/2298208267 learned_addr learned my addr 172.21.15.7:0/2298208267 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca37fe700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b0190 0x7f6ca40c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca3fff700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca40b39b0 0x7f6ca41474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b0190 msgr2=0x7f6ca40c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca2ffd700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b0190 0x7f6ca40c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca40b39b0 msgr2=0x7f6ca41474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca2ffd700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca40b39b0 0x7f6ca41474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.575+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/2298208267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c009ae0 con 0x7f6ca40b14a0 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.576+0000 7f6ca2ffd700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b14a0 0x7f6ca40c8f90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c590 tx=0x7f6c9400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.576+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2298208267 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94011590 con 0x7f6ca40b14a0 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.576+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2298208267 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c940116f0 con 0x7f6ca40b14a0 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.576+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca40c84b0 con 0x7f6ca40b14a0 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.576+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2298208267 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94011ab0 con 0x7f6ca40b14a0 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.576+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca40a56a0 con 0x7f6ca40b14a0 2023-10-07T07:58:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.577+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca40b1b20 con 0x7f6ca40b14a0 2023-10-07T07:58:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.577+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2298208267 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c9401b3b0 con 0x7f6ca40b14a0 2023-10-07T07:58:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.578+0000 7f6c9b7fe700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6c84066f30 0x7f6c840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.578+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2298208267 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c94083ae0 con 0x7f6ca40b14a0 2023-10-07T07:58:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.578+0000 7f6ca37fe700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6c84066f30 0x7f6c840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.578+0000 7f6ca37fe700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6c84066f30 0x7f6c840693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009ca0 tx=0x7f6c8c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.582+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2298208267 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c94050d50 con 0x7f6ca40b14a0 2023-10-07T07:58:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.782+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ca404c3c0 con 0x7f6ca40b14a0 2023-10-07T07:58:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.782+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/2298208267 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c94017020 con 0x7f6ca40b14a0 2023-10-07T07:58:25.783 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-07T07:58:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.784+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6c84066f30 msgr2=0x7f6c840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.784+0000 7f6caa172700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6c84066f30 0x7f6c840693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009ca0 tx=0x7f6c8c008040 comp rx=0 tx=0).stop 2023-10-07T07:58:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.784+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b14a0 msgr2=0x7f6ca40c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.784+0000 7f6caa172700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b14a0 0x7f6ca40c8f90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c590 tx=0x7f6c9400c950 comp rx=0 tx=0).stop 2023-10-07T07:58:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.785+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 shutdown_connections 2023-10-07T07:58:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.785+0000 7f6caa172700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6c84066f30 0x7f6c840693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.785+0000 7f6caa172700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca40b39b0 0x7f6ca41474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.785+0000 7f6caa172700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca40b0190 0x7f6ca40c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.785+0000 7f6caa172700 1 --2- 172.21.15.7:0/2298208267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca40b14a0 0x7f6ca40c8f90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.785+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 >> 172.21.15.7:0/2298208267 conn(0x7f6ca406d350 msgr2=0x7f6ca415ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:25.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.785+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 shutdown_connections 2023-10-07T07:58:25.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.785+0000 7f6caa172700 1 -- 172.21.15.7:0/2298208267 wait complete. 2023-10-07T07:58:25.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T07:58:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.933+0000 7ffb08c17700 1 Processor -- start 2023-10-07T07:58:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.933+0000 7ffb08c17700 1 -- start start 2023-10-07T07:58:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb08c17700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 0x7ffb04156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb08c17700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb040c7a50 0x7ffb0406c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb08c17700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0406b6f0 con 0x7ffb040c75e0 2023-10-07T07:58:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb08c17700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb040cbbf0 con 0x7ffb040c7a50 2023-10-07T07:58:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb08c17700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0406bf70 con 0x7ffb04155d90 2023-10-07T07:58:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb02d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb040c7a50 0x7ffb0406c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb01d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffb040c75e0 0x7ffb0406e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46926/0 (socket says 172.21.15.7:46926) 2023-10-07T07:58:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb01d9b700 1 -- 172.21.15.7:0/1802023727 learned_addr learned my addr 172.21.15.7:0/1802023727 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb0259c700 1 --2- 172.21.15.7:0/1802023727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 0x7ffb04156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb02d9d700 1 -- 172.21.15.7:0/1802023727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 msgr2=0x7ffb04156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb02d9d700 1 --2- 172.21.15.7:0/1802023727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 0x7ffb04156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb02d9d700 1 -- 172.21.15.7:0/1802023727 >> v1:172.21.15.7:6789/0 conn(0x7ffb040c75e0 legacy=0x7ffb0406e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb02d9d700 1 -- 172.21.15.7:0/1802023727 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb040cb910 con 0x7ffb040c7a50 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.934+0000 7ffb0159a700 1 -- 172.21.15.7:0/1802023727 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229254491 0 0) 0x7ffb0406b6f0 con 0x7ffb040c75e0 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb02d9d700 1 --2- 172.21.15.7:0/1802023727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c7a50 0x7ffb0406c400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009e60 tx=0x7ffafc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706e2bb26036b4ab server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb0159a700 1 -- 172.21.15.7:0/1802023727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00e070 con 0x7ffb040c7a50 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb0159a700 1 -- 172.21.15.7:0/1802023727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc00bb60 con 0x7ffb040c7a50 2023-10-07T07:58:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb0159a700 1 -- 172.21.15.7:0/1802023727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc012430 con 0x7ffb040c7a50 2023-10-07T07:58:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb08c17700 1 -- 172.21.15.7:0/1802023727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c7a50 msgr2=0x7ffb0406c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb08c17700 1 --2- 172.21.15.7:0/1802023727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c7a50 0x7ffb0406c400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009e60 tx=0x7ffafc009780 comp rx=0 tx=0).stop 2023-10-07T07:58:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb08c17700 1 -- 172.21.15.7:0/1802023727 shutdown_connections 2023-10-07T07:58:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb08c17700 1 --2- 172.21.15.7:0/1802023727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c7a50 0x7ffb0406c400 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb08c17700 1 --2- 172.21.15.7:0/1802023727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 0x7ffb04156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.935+0000 7ffb08c17700 1 -- 172.21.15.7:0/1802023727 >> 172.21.15.7:0/1802023727 conn(0x7ffb04068100 msgr2=0x7ffb04068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.936+0000 7ffb08c17700 1 -- 172.21.15.7:0/1802023727 shutdown_connections 2023-10-07T07:58:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.936+0000 7ffb08c17700 1 -- 172.21.15.7:0/1802023727 wait complete. 2023-10-07T07:58:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.937+0000 7ffb08c17700 1 Processor -- start 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.937+0000 7ffb08c17700 1 -- start start 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.937+0000 7ffb08c17700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c75e0 0x7ffb040b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.937+0000 7ffb08c17700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb040c7a50 0x7ffb04150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.937+0000 7ffb08c17700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 0x7ffb040c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.937+0000 7ffb08c17700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04158f10 con 0x7ffb040c75e0 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb08c17700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb041566d0 con 0x7ffb040c7a50 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb08c17700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb041569b0 con 0x7ffb04155d90 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb0259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c75e0 0x7ffb040b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb0259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c75e0 0x7ffb040b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46962/0 (socket says 172.21.15.7:46962) 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb0259c700 1 -- 172.21.15.7:0/1140895911 learned_addr learned my addr 172.21.15.7:0/1140895911 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb01d9b700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb040c7a50 0x7ffb04150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb02d9d700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 0x7ffb040c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb0259c700 1 -- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 msgr2=0x7ffb040c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb0259c700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 0x7ffb040c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb0259c700 1 -- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb040c7a50 msgr2=0x7ffb04150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb0259c700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb040c7a50 0x7ffb04150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb0259c700 1 -- 172.21.15.7:0/1140895911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc009a20 con 0x7ffb040c75e0 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb01d9b700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb040c7a50 0x7ffb04150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb0259c700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c75e0 0x7ffb040b26d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e930 tx=0x7ffaf40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffaf27fc700 1 -- 172.21.15.7:0/1140895911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400eed0 con 0x7ffb040c75e0 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffaf27fc700 1 -- 172.21.15.7:0/1140895911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf400c850 con 0x7ffb040c75e0 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb04156c90 con 0x7ffb040c75e0 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffaf27fc700 1 -- 172.21.15.7:0/1140895911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400cbf0 con 0x7ffb040c75e0 2023-10-07T07:58:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.938+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb04157250 con 0x7ffb040c75e0 2023-10-07T07:58:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.939+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb041575c0 con 0x7ffb040c75e0 2023-10-07T07:58:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.940+0000 7ffaf27fc700 1 -- 172.21.15.7:0/1140895911 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffaf4013470 con 0x7ffb040c75e0 2023-10-07T07:58:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.940+0000 7ffaf27fc700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffae4066f30 0x7ffae40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.940+0000 7ffaf27fc700 1 -- 172.21.15.7:0/1140895911 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffaf4084760 con 0x7ffb040c75e0 2023-10-07T07:58:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.941+0000 7ffb01d9b700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffae4066f30 0x7ffae40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.941+0000 7ffb01d9b700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffae4066f30 0x7ffae40693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffb0416a8c0 tx=0x7ffaf8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:25.945+0000 7ffaf27fc700 1 -- 172.21.15.7:0/1140895911 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffaf40519d0 con 0x7ffb040c75e0 2023-10-07T07:58:26.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.146+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb0404c3c0 con 0x7ffb040c75e0 2023-10-07T07:58:26.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.147+0000 7ffaf27fc700 1 -- 172.21.15.7:0/1140895911 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffaf4054ff0 con 0x7ffb040c75e0 2023-10-07T07:58:26.150 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffae4066f30 msgr2=0x7ffae40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffae4066f30 0x7ffae40693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffb0416a8c0 tx=0x7ffaf8009040 comp rx=0 tx=0).stop 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c75e0 msgr2=0x7ffb040b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c75e0 0x7ffb040b26d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e930 tx=0x7ffaf40091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 shutdown_connections 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffae4066f30 0x7ffae40693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb04155d90 0x7ffb040c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb040c75e0 0x7ffb040b26d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 --2- 172.21.15.7:0/1140895911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb040c7a50 0x7ffb04150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 >> 172.21.15.7:0/1140895911 conn(0x7ffb04068100 msgr2=0x7ffb040cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 shutdown_connections 2023-10-07T07:58:26.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.149+0000 7ffb08c17700 1 -- 172.21.15.7:0/1140895911 wait complete. 2023-10-07T07:58:26.163 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:58:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.294+0000 7f8428f45700 1 Processor -- start 2023-10-07T07:58:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.294+0000 7f8428f45700 1 -- start start 2023-10-07T07:58:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.295+0000 7f8428f45700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8424063230 0x7f842406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.295+0000 7f8428f45700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240c9ec0 0x7f842406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.295+0000 7f8428f45700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842406e590 con 0x7f84240c9af0 2023-10-07T07:58:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.295+0000 7f8428f45700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84240b03a0 con 0x7f8424063230 2023-10-07T07:58:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.295+0000 7f8428f45700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842415a330 con 0x7f84240c9ec0 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.295+0000 7f8421d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8424063230 0x7f842406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.295+0000 7f8421d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8424063230 0x7f842406db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46976/0 (socket says 172.21.15.7:46976) 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.295+0000 7f8421d9b700 1 -- 172.21.15.7:0/2170930691 learned_addr learned my addr 172.21.15.7:0/2170930691 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.296+0000 7f8422d9d700 1 --2- 172.21.15.7:0/2170930691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240c9ec0 0x7f842406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.296+0000 7f8421d9b700 1 -- 172.21.15.7:0/2170930691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240c9ec0 msgr2=0x7f842406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.296+0000 7f8421d9b700 1 --2- 172.21.15.7:0/2170930691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240c9ec0 0x7f842406e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.296+0000 7f8421d9b700 1 -- 172.21.15.7:0/2170930691 >> v1:172.21.15.172:6789/0 conn(0x7f84240c9af0 legacy=0x7f842415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.296+0000 7f8421d9b700 1 -- 172.21.15.7:0/2170930691 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842415a610 con 0x7f8424063230 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.296+0000 7f8421d9b700 1 --2- 172.21.15.7:0/2170930691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8424063230 0x7f842406db10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84180099b0 tx=0x7f8418009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19489b5d76ab3f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.296+0000 7f842159a700 1 -- 172.21.15.7:0/2170930691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800b860 con 0x7f8424063230 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.296+0000 7f842159a700 1 -- 172.21.15.7:0/2170930691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841800b9c0 con 0x7f8424063230 2023-10-07T07:58:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.296+0000 7f842159a700 1 -- 172.21.15.7:0/2170930691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800bd60 con 0x7f8424063230 2023-10-07T07:58:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.297+0000 7f8428f45700 1 -- 172.21.15.7:0/2170930691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8424063230 msgr2=0x7f842406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.297+0000 7f8428f45700 1 --2- 172.21.15.7:0/2170930691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8424063230 0x7f842406db10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84180099b0 tx=0x7f8418009250 comp rx=0 tx=0).stop 2023-10-07T07:58:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.297+0000 7f8428f45700 1 -- 172.21.15.7:0/2170930691 shutdown_connections 2023-10-07T07:58:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.297+0000 7f8428f45700 1 --2- 172.21.15.7:0/2170930691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84240c9ec0 0x7f842406e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.297+0000 7f8428f45700 1 --2- 172.21.15.7:0/2170930691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8424063230 0x7f842406db10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.297+0000 7f8428f45700 1 -- 172.21.15.7:0/2170930691 >> 172.21.15.7:0/2170930691 conn(0x7f84240cd1a0 msgr2=0x7f84240c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.297+0000 7f8428f45700 1 -- 172.21.15.7:0/2170930691 shutdown_connections 2023-10-07T07:58:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.297+0000 7f8428f45700 1 -- 172.21.15.7:0/2170930691 wait complete. 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.298+0000 7f8428f45700 1 Processor -- start 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.298+0000 7f8428f45700 1 -- start start 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8428f45700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8424063230 0x7f8424148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8428f45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84240c9af0 0x7f84240cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8428f45700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84240c9ec0 0x7f84240cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8428f45700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842416a520 con 0x7f84240c9af0 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8428f45700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8424169bf0 con 0x7f84240c9ec0 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8428f45700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8424169ed0 con 0x7f8424063230 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8421d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84240c9af0 0x7f84240cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8421d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84240c9af0 0x7f84240cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46992/0 (socket says 172.21.15.7:46992) 2023-10-07T07:58:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8421d9b700 1 -- 172.21.15.7:0/1556776349 learned_addr learned my addr 172.21.15.7:0/1556776349 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f842259c700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8424063230 0x7f8424148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8422d9d700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84240c9ec0 0x7f84240cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8421d9b700 1 -- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8424063230 msgr2=0x7f8424148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8421d9b700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8424063230 0x7f8424148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8421d9b700 1 -- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84240c9ec0 msgr2=0x7f84240cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8421d9b700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84240c9ec0 0x7f84240cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8421d9b700 1 -- 172.21.15.7:0/1556776349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180094f0 con 0x7f84240c9af0 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.299+0000 7f8421d9b700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84240c9af0 0x7f84240cb7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f842406d7c0 tx=0x7f8418017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.300+0000 7f84127fc700 1 -- 172.21.15.7:0/1556776349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418024440 con 0x7f84240c9af0 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.300+0000 7f84127fc700 1 -- 172.21.15.7:0/1556776349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84180245c0 con 0x7f84240c9af0 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.300+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842416a1b0 con 0x7f84240c9af0 2023-10-07T07:58:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.300+0000 7f84127fc700 1 -- 172.21.15.7:0/1556776349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418024980 con 0x7f84240c9af0 2023-10-07T07:58:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.300+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84241698a0 con 0x7f84240c9af0 2023-10-07T07:58:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.301+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84240716c0 con 0x7f84240c9af0 2023-10-07T07:58:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.301+0000 7f84127fc700 1 -- 172.21.15.7:0/1556776349 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f841801a420 con 0x7f84240c9af0 2023-10-07T07:58:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.302+0000 7f84127fc700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8404066f30 0x7f84040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.302+0000 7f84127fc700 1 -- 172.21.15.7:0/1556776349 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8418086350 con 0x7f84240c9af0 2023-10-07T07:58:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.302+0000 7f842259c700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8404066f30 0x7f84040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.303+0000 7f842259c700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8404066f30 0x7f84040693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8414003860 tx=0x7f8414009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:26.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.306+0000 7f84127fc700 1 -- 172.21.15.7:0/1556776349 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84180535c0 con 0x7f84240c9af0 2023-10-07T07:58:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.555+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8424068230 con 0x7f84240c9af0 2023-10-07T07:58:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.555+0000 7f84127fc700 1 -- 172.21.15.7:0/1556776349 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f8418056be0 con 0x7f84240c9af0 2023-10-07T07:58:26.557 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:26.557 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2318752844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2318752844},{"type":"v1","addr":"172.21.15.172:6836","nonce":2318752844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1231430544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1231430544},{"type":"v1","addr":"172.21.15.172:6837","nonce":1231430544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1421504798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1421504798},{"type":"v1","addr":"172.21.15.7:6833","nonce":1421504798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:57:50.139330+0000","modified":"2023-10-07T07:57:53.727131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T07:58:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8404066f30 msgr2=0x7f84040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8404066f30 0x7f84040693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8414003860 tx=0x7f8414009040 comp rx=0 tx=0).stop 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84240c9af0 msgr2=0x7f84240cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84240c9af0 0x7f84240cb7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f842406d7c0 tx=0x7f8418017000 comp rx=0 tx=0).stop 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 shutdown_connections 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8404066f30 0x7f84040693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84240c9ec0 0x7f84240cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8424063230 0x7f8424148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 --2- 172.21.15.7:0/1556776349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84240c9af0 0x7f84240cb7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 >> 172.21.15.7:0/1556776349 conn(0x7f84240cd1a0 msgr2=0x7f84241575e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 shutdown_connections 2023-10-07T07:58:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.558+0000 7f8428f45700 1 -- 172.21.15.7:0/1556776349 wait complete. 2023-10-07T07:58:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2023-10-07T07:58:26.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:26.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.703+0000 7f31be8fa700 1 Processor -- start 2023-10-07T07:58:26.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.703+0000 7f31be8fa700 1 -- start start 2023-10-07T07:58:26.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31be8fa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31b8155f20 0x7f31b80c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31be8fa700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f31b81562f0 0x7f31b806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31be8fa700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b806b700 con 0x7f31b806e2c0 2023-10-07T07:58:26.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31be8fa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b806aaf0 con 0x7f31b8155f20 2023-10-07T07:58:26.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31be8fa700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b806add0 con 0x7f31b81562f0 2023-10-07T07:58:26.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31b7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31b8155f20 0x7f31b80c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31b7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31b8155f20 0x7f31b80c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46994/0 (socket says 172.21.15.7:46994) 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31b7fff700 1 -- 172.21.15.7:0/2415695568 learned_addr learned my addr 172.21.15.7:0/2415695568 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31b77fe700 1 --2- 172.21.15.7:0/2415695568 >> v2:172.21.15.172:3300/0 conn(0x7f31b81562f0 0x7f31b806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31b7fff700 1 -- 172.21.15.7:0/2415695568 >> v2:172.21.15.172:3300/0 conn(0x7f31b81562f0 msgr2=0x7f31b806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31b7fff700 1 --2- 172.21.15.7:0/2415695568 >> v2:172.21.15.172:3300/0 conn(0x7f31b81562f0 0x7f31b806a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.704+0000 7f31b7fff700 1 -- 172.21.15.7:0/2415695568 >> v1:172.21.15.172:6789/0 conn(0x7f31b806e2c0 legacy=0x7f31b806e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31b7fff700 1 -- 172.21.15.7:0/2415695568 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b806b0b0 con 0x7f31b8155f20 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31b77fe700 1 --2- 172.21.15.7:0/2415695568 >> v2:172.21.15.172:3300/0 conn(0x7f31b81562f0 0x7f31b806a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31b7fff700 1 --2- 172.21.15.7:0/2415695568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b8155f20 0x7f31b80c92b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f31b806e0b0 tx=0x7f31a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d951665f78ede582 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31b6ffd700 1 -- 172.21.15.7:0/2415695568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a800b860 con 0x7f31b8155f20 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31b6ffd700 1 -- 172.21.15.7:0/2415695568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a800b9c0 con 0x7f31b8155f20 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31b6ffd700 1 -- 172.21.15.7:0/2415695568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a800bd60 con 0x7f31b8155f20 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31be8fa700 1 -- 172.21.15.7:0/2415695568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b8155f20 msgr2=0x7f31b80c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31be8fa700 1 --2- 172.21.15.7:0/2415695568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b8155f20 0x7f31b80c92b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f31b806e0b0 tx=0x7f31a8009250 comp rx=0 tx=0).stop 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31be8fa700 1 -- 172.21.15.7:0/2415695568 shutdown_connections 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31be8fa700 1 --2- 172.21.15.7:0/2415695568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b8155f20 0x7f31b80c92b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31be8fa700 1 --2- 172.21.15.7:0/2415695568 >> v2:172.21.15.172:3300/0 conn(0x7f31b81562f0 0x7f31b806a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.705+0000 7f31be8fa700 1 -- 172.21.15.7:0/2415695568 >> 172.21.15.7:0/2415695568 conn(0x7f31b8068bd0 msgr2=0x7f31b80ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.706+0000 7f31be8fa700 1 -- 172.21.15.7:0/2415695568 shutdown_connections 2023-10-07T07:58:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.706+0000 7f31be8fa700 1 -- 172.21.15.7:0/2415695568 wait complete. 2023-10-07T07:58:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.707+0000 7f31be8fa700 1 Processor -- start 2023-10-07T07:58:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.707+0000 7f31be8fa700 1 -- start start 2023-10-07T07:58:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31be8fa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31b806e2c0 0x7f31b8146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31be8fa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31b8155f20 0x7f31b80aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31be8fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b81562f0 0x7f31b81502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31be8fa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b814d110 con 0x7f31b81562f0 2023-10-07T07:58:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31be8fa700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80c6b70 con 0x7f31b8155f20 2023-10-07T07:58:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31be8fa700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80c6e50 con 0x7f31b806e2c0 2023-10-07T07:58:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31bce97700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b81562f0 0x7f31b81502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31bce97700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b81562f0 0x7f31b81502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47004/0 (socket says 172.21.15.7:47004) 2023-10-07T07:58:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31bce97700 1 -- 172.21.15.7:0/895820468 learned_addr learned my addr 172.21.15.7:0/895820468 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31b77fe700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31b8155f20 0x7f31b80aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31b7fff700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31b806e2c0 0x7f31b8146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31bce97700 1 -- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31b806e2c0 msgr2=0x7f31b8146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31bce97700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31b806e2c0 0x7f31b8146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31bce97700 1 -- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31b8155f20 msgr2=0x7f31b80aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31bce97700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31b8155f20 0x7f31b80aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31bce97700 1 -- 172.21.15.7:0/895820468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a80094f0 con 0x7f31b81562f0 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.708+0000 7f31bce97700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b81562f0 0x7f31b81502d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f31b000c8e0 tx=0x7f31b0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.709+0000 7f31affff700 1 -- 172.21.15.7:0/895820468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0012630 con 0x7f31b81562f0 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.709+0000 7f31affff700 1 -- 172.21.15.7:0/895820468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b0012790 con 0x7f31b81562f0 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.709+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b814cac0 con 0x7f31b81562f0 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.709+0000 7f31affff700 1 -- 172.21.15.7:0/895820468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0012b50 con 0x7f31b81562f0 2023-10-07T07:58:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.709+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b81470c0 con 0x7f31b81562f0 2023-10-07T07:58:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.710+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b80716c0 con 0x7f31b81562f0 2023-10-07T07:58:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.710+0000 7f31affff700 1 -- 172.21.15.7:0/895820468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31b001a3a0 con 0x7f31b81562f0 2023-10-07T07:58:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.710+0000 7f31affff700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3198066f30 0x7f31980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.711+0000 7f31affff700 1 -- 172.21.15.7:0/895820468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31b0084740 con 0x7f31b81562f0 2023-10-07T07:58:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.711+0000 7f31b7fff700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3198066f30 0x7f31980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.711+0000 7f31b7fff700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3198066f30 0x7f31980693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f31b806e0b0 tx=0x7f31a8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.715+0000 7f31affff700 1 -- 172.21.15.7:0/895820468 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31b00519b0 con 0x7f31b81562f0 2023-10-07T07:58:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.915+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31b806f1f0 con 0x7f31b81562f0 2023-10-07T07:58:26.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.916+0000 7f31affff700 1 -- 172.21.15.7:0/895820468 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f31b0054fd0 con 0x7f31b81562f0 2023-10-07T07:58:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:26.917 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:26.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3198066f30 msgr2=0x7f31980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3198066f30 0x7f31980693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f31b806e0b0 tx=0x7f31a8017000 comp rx=0 tx=0).stop 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b81562f0 msgr2=0x7f31b81502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b81562f0 0x7f31b81502d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f31b000c8e0 tx=0x7f31b0001710 comp rx=0 tx=0).stop 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 shutdown_connections 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3198066f30 0x7f31980693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b81562f0 0x7f31b81502d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31b806e2c0 0x7f31b8146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 --2- 172.21.15.7:0/895820468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31b8155f20 0x7f31b80aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 >> 172.21.15.7:0/895820468 conn(0x7f31b8068bd0 msgr2=0x7f31b81519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.918+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 shutdown_connections 2023-10-07T07:58:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:26.919+0000 7f31be8fa700 1 -- 172.21.15.7:0/895820468 wait complete. 2023-10-07T07:58:26.933 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-07T07:58:26.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:58:27.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.064+0000 7f209f757700 1 Processor -- start 2023-10-07T07:58:27.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.065+0000 7f209f757700 1 -- start start 2023-10-07T07:58:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209f757700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f20980aaa50 0x7f2098164110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209f757700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209806bb70 con 0x7f20981592c0 2023-10-07T07:58:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209f757700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20980cda10 con 0x7f2098159690 2023-10-07T07:58:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209f757700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209806af60 con 0x7f20980aaa50 2023-10-07T07:58:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209d4f3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f20981592c0 0x7f209806a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46940/0 (socket says 172.21.15.7:46940) 2023-10-07T07:58:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209d4f3700 1 -- 172.21.15.7:0/3582864965 learned_addr learned my addr 172.21.15.7:0/3582864965 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209dcf4700 1 --2- 172.21.15.7:0/3582864965 >> v2:172.21.15.172:3300/0 conn(0x7f20980aaa50 0x7f2098164110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f208ffff700 1 -- 172.21.15.7:0/3582864965 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610216120 0 0) 0x7f209806bb70 con 0x7f20981592c0 2023-10-07T07:58:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f208ffff700 1 -- 172.21.15.7:0/3582864965 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2084003570 con 0x7f20981592c0 2023-10-07T07:58:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f208ffff700 1 -- 172.21.15.7:0/3582864965 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683055949 0 0) 0x7f20980cda10 con 0x7f2098159690 2023-10-07T07:58:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f208ffff700 1 -- 172.21.15.7:0/3582864965 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f209806bb70 con 0x7f2098159690 2023-10-07T07:58:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209dcf4700 1 -- 172.21.15.7:0/3582864965 >> v1:172.21.15.172:6789/0 conn(0x7f2098159690 legacy=0x7f20980cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209dcf4700 1 -- 172.21.15.7:0/3582864965 >> v1:172.21.15.7:6789/0 conn(0x7f20981592c0 legacy=0x7f209806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.066+0000 7f209dcf4700 1 -- 172.21.15.7:0/3582864965 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209806b240 con 0x7f20980aaa50 2023-10-07T07:58:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f209dcf4700 1 --2- 172.21.15.7:0/3582864965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20980aaa50 0x7f2098164110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2090009e60 tx=0x7f2090009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8aae574f4d49125 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f208ffff700 1 -- 172.21.15.7:0/3582864965 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209000e070 con 0x7f20980aaa50 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f208ffff700 1 -- 172.21.15.7:0/3582864965 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209000bab0 con 0x7f20980aaa50 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f208ffff700 1 -- 172.21.15.7:0/3582864965 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209000be70 con 0x7f20980aaa50 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f209f757700 1 -- 172.21.15.7:0/3582864965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20980aaa50 msgr2=0x7f2098164110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f209f757700 1 --2- 172.21.15.7:0/3582864965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20980aaa50 0x7f2098164110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2090009e60 tx=0x7f2090009780 comp rx=0 tx=0).stop 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f209f757700 1 -- 172.21.15.7:0/3582864965 shutdown_connections 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f209f757700 1 --2- 172.21.15.7:0/3582864965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20980aaa50 0x7f2098164110 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f209f757700 1 -- 172.21.15.7:0/3582864965 >> 172.21.15.7:0/3582864965 conn(0x7f2098068bd0 msgr2=0x7f20980ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f209f757700 1 -- 172.21.15.7:0/3582864965 shutdown_connections 2023-10-07T07:58:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.067+0000 7f209f757700 1 -- 172.21.15.7:0/3582864965 wait complete. 2023-10-07T07:58:27.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.069+0000 7f209f757700 1 Processor -- start 2023-10-07T07:58:27.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.069+0000 7f209f757700 1 -- start start 2023-10-07T07:58:27.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209f757700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20980aaa50 0x7f2098154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209f757700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20981592c0 0x7f209815d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209f757700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2098159690 0x7f209815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209f757700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209815db50 con 0x7f20980aaa50 2023-10-07T07:58:27.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209f757700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209815de30 con 0x7f20981592c0 2023-10-07T07:58:27.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209f757700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209815e110 con 0x7f2098159690 2023-10-07T07:58:27.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209d4f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20980aaa50 0x7f2098154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209d4f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20980aaa50 0x7f2098154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47020/0 (socket says 172.21.15.7:47020) 2023-10-07T07:58:27.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209d4f3700 1 -- 172.21.15.7:0/4136142876 learned_addr learned my addr 172.21.15.7:0/4136142876 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:27.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209dcf4700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2098159690 0x7f209815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209ccf2700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20981592c0 0x7f209815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209d4f3700 1 -- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2098159690 msgr2=0x7f209815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209d4f3700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2098159690 0x7f209815f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209d4f3700 1 -- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20981592c0 msgr2=0x7f209815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209d4f3700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20981592c0 0x7f209815d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209d4f3700 1 -- 172.21.15.7:0/4136142876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090009a20 con 0x7f20980aaa50 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f209d4f3700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20980aaa50 0x7f2098154bc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f209400c790 tx=0x7f2094007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.070+0000 7f208d7fa700 1 -- 172.21.15.7:0/4136142876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094005b30 con 0x7f20980aaa50 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.071+0000 7f208d7fa700 1 -- 172.21.15.7:0/4136142876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2094005c90 con 0x7f20980aaa50 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.071+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209815e3f0 con 0x7f20980aaa50 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.071+0000 7f208d7fa700 1 -- 172.21.15.7:0/4136142876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094007300 con 0x7f20980aaa50 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.071+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209815d5a0 con 0x7f20980aaa50 2023-10-07T07:58:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.072+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209815e910 con 0x7f20980aaa50 2023-10-07T07:58:27.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.072+0000 7f208d7fa700 1 -- 172.21.15.7:0/4136142876 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f209401b400 con 0x7f20980aaa50 2023-10-07T07:58:27.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.073+0000 7f208d7fa700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f207c066f30 0x7f207c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.073+0000 7f208d7fa700 1 -- 172.21.15.7:0/4136142876 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f20940840d0 con 0x7f20980aaa50 2023-10-07T07:58:27.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.073+0000 7f209ccf2700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f207c066f30 0x7f207c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.073+0000 7f209ccf2700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f207c066f30 0x7f207c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f209815cf70 tx=0x7f2088009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:27.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.077+0000 7f208d7fa700 1 -- 172.21.15.7:0/4136142876 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2094054e80 con 0x7f20980aaa50 2023-10-07T07:58:27.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.323+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f209815b3b0 con 0x7f20980aaa50 2023-10-07T07:58:27.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.323+0000 7f208d7fa700 1 -- 172.21.15.7:0/4136142876 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f20940076f0 con 0x7f20980aaa50 2023-10-07T07:58:27.324 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:27.324 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2318752844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2318752844},{"type":"v1","addr":"172.21.15.172:6836","nonce":2318752844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1231430544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1231430544},{"type":"v1","addr":"172.21.15.172:6837","nonce":1231430544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1421504798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1421504798},{"type":"v1","addr":"172.21.15.7:6833","nonce":1421504798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:57:50.139330+0000","modified":"2023-10-07T07:57:53.727131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T07:58:27.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.325+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f207c066f30 msgr2=0x7f207c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.325+0000 7f209f757700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f207c066f30 0x7f207c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f209815cf70 tx=0x7f2088009040 comp rx=0 tx=0).stop 2023-10-07T07:58:27.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.325+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20980aaa50 msgr2=0x7f2098154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.325+0000 7f209f757700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20980aaa50 0x7f2098154bc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f209400c790 tx=0x7f2094007ea0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.326+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 shutdown_connections 2023-10-07T07:58:27.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.326+0000 7f209f757700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f207c066f30 0x7f207c0693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.326+0000 7f209f757700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2098159690 0x7f209815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.326+0000 7f209f757700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20980aaa50 0x7f2098154bc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.326+0000 7f209f757700 1 --2- 172.21.15.7:0/4136142876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20981592c0 0x7f209815d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.326+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 >> 172.21.15.7:0/4136142876 conn(0x7f2098068bd0 msgr2=0x7f20980c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:27.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.326+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 shutdown_connections 2023-10-07T07:58:27.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.326+0000 7f209f757700 1 -- 172.21.15.7:0/4136142876 wait complete. 2023-10-07T07:58:27.328 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2023-10-07T07:58:27.340 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:58:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.471+0000 7f7ed3ab1700 1 Processor -- start 2023-10-07T07:58:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.472+0000 7f7ed3ab1700 1 -- start start 2023-10-07T07:58:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed3ab1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7ecc1570a0 0x7f7ecc0c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed3ab1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc157470 0x7f7ecc06b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed3ab1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ecc06bb60 0x7f7ecc06cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed3ab1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ecc06bfe0 con 0x7f7ecc06bb60 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed3ab1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ecc0c3b80 con 0x7f7ecc1570a0 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed3ab1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0c2f70 con 0x7f7ecc157470 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed204e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ecc06bb60 0x7f7ecc06cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed204e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ecc06bb60 0x7f7ecc06cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47036/0 (socket says 172.21.15.7:47036) 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed204e700 1 -- 172.21.15.7:0/3530263542 learned_addr learned my addr 172.21.15.7:0/3530263542 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed184d700 1 --2- 172.21.15.7:0/3530263542 >> v2:172.21.15.172:3300/0 conn(0x7f7ecc1570a0 0x7f7ecc0c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed104c700 1 --2- 172.21.15.7:0/3530263542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc157470 0x7f7ecc06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed204e700 1 -- 172.21.15.7:0/3530263542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc157470 msgr2=0x7f7ecc06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed204e700 1 --2- 172.21.15.7:0/3530263542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc157470 0x7f7ecc06b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed204e700 1 -- 172.21.15.7:0/3530263542 >> v2:172.21.15.172:3300/0 conn(0x7f7ecc1570a0 msgr2=0x7f7ecc0c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed204e700 1 --2- 172.21.15.7:0/3530263542 >> v2:172.21.15.172:3300/0 conn(0x7f7ecc1570a0 0x7f7ecc0c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed204e700 1 -- 172.21.15.7:0/3530263542 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc0c3250 con 0x7f7ecc06bb60 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed204e700 1 --2- 172.21.15.7:0/3530263542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06bb60 0x7f7ecc06cac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0caa70 tx=0x7f7ec4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22812ef3145e91ef server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.473+0000 7f7ed084b700 1 -- 172.21.15.7:0/3530263542 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400e070 con 0x7f7ecc06bb60 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed084b700 1 -- 172.21.15.7:0/3530263542 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec400ba50 con 0x7f7ecc06bb60 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed084b700 1 -- 172.21.15.7:0/3530263542 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec40092e0 con 0x7f7ecc06bb60 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3530263542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06bb60 msgr2=0x7f7ecc06cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3530263542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06bb60 0x7f7ecc06cac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0caa70 tx=0x7f7ec4009840 comp rx=0 tx=0).stop 2023-10-07T07:58:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3530263542 shutdown_connections 2023-10-07T07:58:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3530263542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06bb60 0x7f7ecc06cac0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3530263542 >> v2:172.21.15.172:3300/0 conn(0x7f7ecc1570a0 0x7f7ecc0c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3530263542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc157470 0x7f7ecc06b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3530263542 >> 172.21.15.7:0/3530263542 conn(0x7f7ecc067ac0 msgr2=0x7f7ecc0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3530263542 shutdown_connections 2023-10-07T07:58:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.474+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3530263542 wait complete. 2023-10-07T07:58:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.475+0000 7f7ed3ab1700 1 Processor -- start 2023-10-07T07:58:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed3ab1700 1 -- start start 2023-10-07T07:58:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed3ab1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc06bb60 0x7f7ecc152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed3ab1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc1570a0 0x7f7ecc0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed3ab1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc157470 0x7f7ecc13ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed3ab1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0b2330 con 0x7f7ecc157470 2023-10-07T07:58:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed3ab1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0b27c0 con 0x7f7ecc1570a0 2023-10-07T07:58:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed3ab1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0b1720 con 0x7f7ecc06bb60 2023-10-07T07:58:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed204e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc157470 0x7f7ecc13ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed204e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc157470 0x7f7ecc13ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47052/0 (socket says 172.21.15.7:47052) 2023-10-07T07:58:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.476+0000 7f7ed204e700 1 -- 172.21.15.7:0/3334306133 learned_addr learned my addr 172.21.15.7:0/3334306133 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed104c700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc1570a0 0x7f7ecc0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed184d700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc06bb60 0x7f7ecc152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed204e700 1 -- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc06bb60 msgr2=0x7f7ecc152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed204e700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc06bb60 0x7f7ecc152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed204e700 1 -- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc1570a0 msgr2=0x7f7ecc0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed204e700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc1570a0 0x7f7ecc0b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed204e700 1 -- 172.21.15.7:0/3334306133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec4009ae0 con 0x7f7ecc157470 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed184d700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc06bb60 0x7f7ecc152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed204e700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc157470 0x7f7ecc13ecc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0cbc20 tx=0x7f7ec4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ec1ffb700 1 -- 172.21.15.7:0/3334306133 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec40032f0 con 0x7f7ecc157470 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ec1ffb700 1 -- 172.21.15.7:0/3334306133 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec4008dd0 con 0x7f7ecc157470 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ec1ffb700 1 -- 172.21.15.7:0/3334306133 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec40256b0 con 0x7f7ecc157470 2023-10-07T07:58:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc0b1a00 con 0x7f7ecc157470 2023-10-07T07:58:27.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.477+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc0b1fc0 con 0x7f7ecc157470 2023-10-07T07:58:27.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.478+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc0716c0 con 0x7f7ecc157470 2023-10-07T07:58:27.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.479+0000 7f7ec1ffb700 1 -- 172.21.15.7:0/3334306133 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ec4003040 con 0x7f7ecc157470 2023-10-07T07:58:27.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.479+0000 7f7ec1ffb700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7eb0066f30 0x7f7eb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.479+0000 7f7ec1ffb700 1 -- 172.21.15.7:0/3334306133 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ec4085bb0 con 0x7f7ecc157470 2023-10-07T07:58:27.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.480+0000 7f7ed184d700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7eb0066f30 0x7f7eb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.480+0000 7f7ed184d700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7eb0066f30 0x7f7eb00693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8006fd0 tx=0x7f7ec8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:27.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.484+0000 7f7ec1ffb700 1 -- 172.21.15.7:0/3334306133 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ec4052e30 con 0x7f7ecc157470 2023-10-07T07:58:27.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.737+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ecc13a9c0 con 0x7f7ecc157470 2023-10-07T07:58:27.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.737+0000 7f7ec1ffb700 1 -- 172.21.15.7:0/3334306133 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f7ec4009940 con 0x7f7ecc157470 2023-10-07T07:58:27.738 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:27.738 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2318752844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2318752844},{"type":"v1","addr":"172.21.15.172:6836","nonce":2318752844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1231430544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1231430544},{"type":"v1","addr":"172.21.15.172:6837","nonce":1231430544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1421504798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1421504798},{"type":"v1","addr":"172.21.15.7:6833","nonce":1421504798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:57:50.139330+0000","modified":"2023-10-07T07:57:53.727131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1878634661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1878634661},{"type":"v1","addr":"172.21.15.7:6835","nonce":1878634661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T07:58:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7eb0066f30 msgr2=0x7f7eb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7eb0066f30 0x7f7eb00693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8006fd0 tx=0x7f7ec8008040 comp rx=0 tx=0).stop 2023-10-07T07:58:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc157470 msgr2=0x7f7ecc13ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc157470 0x7f7ecc13ecc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0cbc20 tx=0x7f7ec4018000 comp rx=0 tx=0).stop 2023-10-07T07:58:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 shutdown_connections 2023-10-07T07:58:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7eb0066f30 0x7f7eb00693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc157470 0x7f7ecc13ecc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ecc06bb60 0x7f7ecc152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 --2- 172.21.15.7:0/3334306133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc1570a0 0x7f7ecc0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 >> 172.21.15.7:0/3334306133 conn(0x7f7ecc067ac0 msgr2=0x7f7ecc139af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 shutdown_connections 2023-10-07T07:58:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.739+0000 7f7ed3ab1700 1 -- 172.21.15.7:0/3334306133 wait complete. 2023-10-07T07:58:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2023-10-07T07:58:27.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:27.756 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:58:27.758 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:58:27.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.888+0000 7f74e6fff700 1 Processor -- start 2023-10-07T07:58:27.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.888+0000 7f74e6fff700 1 -- start start 2023-10-07T07:58:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e6fff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f74e01592c0 0x7f74e006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e6fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74e00cb290 0x7f74e006a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e6fff700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e006bb70 con 0x7f74e0159690 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e6fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e00cda10 con 0x7f74e00cb290 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e6fff700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e006af60 con 0x7f74e01592c0 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e559c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74e00cb290 0x7f74e006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e559c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74e00cb290 0x7f74e006a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47066/0 (socket says 172.21.15.7:47066) 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e559c700 1 -- 172.21.15.7:0/3200449336 learned_addr learned my addr 172.21.15.7:0/3200449336 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e4d9b700 1 --2- 172.21.15.7:0/3200449336 >> v2:172.21.15.172:3300/0 conn(0x7f74e01592c0 0x7f74e006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e559c700 1 -- 172.21.15.7:0/3200449336 >> v2:172.21.15.172:3300/0 conn(0x7f74e01592c0 msgr2=0x7f74e006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e559c700 1 --2- 172.21.15.7:0/3200449336 >> v2:172.21.15.172:3300/0 conn(0x7f74e01592c0 0x7f74e006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e559c700 1 -- 172.21.15.7:0/3200449336 >> v1:172.21.15.172:6789/0 conn(0x7f74e0159690 legacy=0x7f74e00cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e559c700 1 -- 172.21.15.7:0/3200449336 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e006b240 con 0x7f74e00cb290 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.889+0000 7f74e559c700 1 --2- 172.21.15.7:0/3200449336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e00cb290 0x7f74e006a930 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f74e006daf0 tx=0x7f74d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6aa2f39c37c073 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74df7fe700 1 -- 172.21.15.7:0/3200449336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d800e070 con 0x7f74e00cb290 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74df7fe700 1 -- 172.21.15.7:0/3200449336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d800ba20 con 0x7f74e00cb290 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74df7fe700 1 -- 172.21.15.7:0/3200449336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80092e0 con 0x7f74e00cb290 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74e6fff700 1 -- 172.21.15.7:0/3200449336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e00cb290 msgr2=0x7f74e006a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74e6fff700 1 --2- 172.21.15.7:0/3200449336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e00cb290 0x7f74e006a930 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f74e006daf0 tx=0x7f74d8009840 comp rx=0 tx=0).stop 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74e6fff700 1 -- 172.21.15.7:0/3200449336 shutdown_connections 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74e6fff700 1 --2- 172.21.15.7:0/3200449336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e00cb290 0x7f74e006a930 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74e6fff700 1 --2- 172.21.15.7:0/3200449336 >> v2:172.21.15.172:3300/0 conn(0x7f74e01592c0 0x7f74e006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74e6fff700 1 -- 172.21.15.7:0/3200449336 >> 172.21.15.7:0/3200449336 conn(0x7f74e0068bd0 msgr2=0x7f74e00ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74e6fff700 1 -- 172.21.15.7:0/3200449336 shutdown_connections 2023-10-07T07:58:27.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.890+0000 7f74e6fff700 1 -- 172.21.15.7:0/3200449336 wait complete. 2023-10-07T07:58:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.892+0000 7f74e6fff700 1 Processor -- start 2023-10-07T07:58:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.892+0000 7f74e6fff700 1 -- start start 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.892+0000 7f74e6fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74e00cb290 0x7f74e00c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.892+0000 7f74e6fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e01592c0 0x7f74e0155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74e6fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74e0159690 0x7f74e00ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74e6fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00c6800 con 0x7f74e01592c0 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74e6fff700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00ae280 con 0x7f74e00cb290 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74e6fff700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0155e60 con 0x7f74e0159690 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e01592c0 0x7f74e0155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e01592c0 0x7f74e0155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47082/0 (socket says 172.21.15.7:47082) 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dffff700 1 -- 172.21.15.7:0/351174191 learned_addr learned my addr 172.21.15.7:0/351174191 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74e4d9b700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74e00cb290 0x7f74e00c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74e559c700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74e0159690 0x7f74e00ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dffff700 1 -- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74e0159690 msgr2=0x7f74e00ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dffff700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74e0159690 0x7f74e00ad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dffff700 1 -- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74e00cb290 msgr2=0x7f74e00c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dffff700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74e00cb290 0x7f74e00c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dffff700 1 -- 172.21.15.7:0/351174191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d8009ae0 con 0x7f74e01592c0 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74e559c700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74e0159690 0x7f74e00ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dffff700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e01592c0 0x7f74e0155920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00c590 tx=0x7f74cc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dcff9700 1 -- 172.21.15.7:0/351174191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc011590 con 0x7f74e01592c0 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dcff9700 1 -- 172.21.15.7:0/351174191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc011710 con 0x7f74e01592c0 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e00ad9e0 con 0x7f74e01592c0 2023-10-07T07:58:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.893+0000 7f74dcff9700 1 -- 172.21.15.7:0/351174191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc011ad0 con 0x7f74e01592c0 2023-10-07T07:58:27.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.894+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e00adfa0 con 0x7f74e01592c0 2023-10-07T07:58:27.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.894+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e00716c0 con 0x7f74e01592c0 2023-10-07T07:58:27.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.895+0000 7f74dcff9700 1 -- 172.21.15.7:0/351174191 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74cc01b3b0 con 0x7f74e01592c0 2023-10-07T07:58:27.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.895+0000 7f74dcff9700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f74c4066f30 0x7f74c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:27.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.895+0000 7f74dcff9700 1 -- 172.21.15.7:0/351174191 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74cc083b10 con 0x7f74e01592c0 2023-10-07T07:58:27.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.896+0000 7f74e4d9b700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f74c4066f30 0x7f74c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:27.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.896+0000 7f74e4d9b700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f74c4066f30 0x7f74c40693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009db0 tx=0x7f74d40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:27.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:27.900+0000 7f74dcff9700 1 -- 172.21.15.7:0/351174191 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74cc050ce0 con 0x7f74e01592c0 2023-10-07T07:58:28.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.104+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74e006e040 con 0x7f74e01592c0 2023-10-07T07:58:28.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.105+0000 7f74dcff9700 1 -- 172.21.15.7:0/351174191 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f74cc017070 con 0x7f74e01592c0 2023-10-07T07:58:28.106 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:28.106 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:57:51.715134+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:57:48.285077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T07:57:49.111979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:28.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f74c4066f30 msgr2=0x7f74c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f74c4066f30 0x7f74c40693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009db0 tx=0x7f74d40091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e01592c0 msgr2=0x7f74e0155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e01592c0 0x7f74e0155920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00c590 tx=0x7f74cc00c8a0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 shutdown_connections 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f74c4066f30 0x7f74c40693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74e0159690 0x7f74e00ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74e00cb290 0x7f74e00c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 --2- 172.21.15.7:0/351174191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e01592c0 0x7f74e0155920 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 >> 172.21.15.7:0/351174191 conn(0x7f74e0068bd0 msgr2=0x7f74e006a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 shutdown_connections 2023-10-07T07:58:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.107+0000 7f74e6fff700 1 -- 172.21.15.7:0/351174191 wait complete. 2023-10-07T07:58:28.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-07T07:58:28.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.255+0000 7f209b253700 1 Processor -- start 2023-10-07T07:58:28.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.255+0000 7f209b253700 1 -- start start 2023-10-07T07:58:28.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.256+0000 7f209b253700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20940c9af0 0x7f209415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.256+0000 7f209b253700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209415a330 con 0x7f20940c9ec0 2023-10-07T07:58:28.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.256+0000 7f209b253700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209415a610 con 0x7f2094063230 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.256+0000 7f209b253700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20940b03a0 con 0x7f20940c9af0 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.256+0000 7f20997f0700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f20940c9ec0 0x7f2094158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46942/0 (socket says 172.21.15.7:46942) 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.256+0000 7f20997f0700 1 -- 172.21.15.7:0/1095873434 learned_addr learned my addr 172.21.15.7:0/1095873434 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f2098fef700 1 --2- 172.21.15.7:0/1095873434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20940c9af0 0x7f209415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f20937fe700 1 -- 172.21.15.7:0/1095873434 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061602093 0 0) 0x7f209415a330 con 0x7f20940c9ec0 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f20937fe700 1 -- 172.21.15.7:0/1095873434 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2080003570 con 0x7f20940c9ec0 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f20937fe700 1 -- 172.21.15.7:0/1095873434 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839485192 0 0) 0x7f209415a610 con 0x7f2094063230 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f20937fe700 1 -- 172.21.15.7:0/1095873434 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f209415a330 con 0x7f2094063230 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f2098fef700 1 -- 172.21.15.7:0/1095873434 >> v1:172.21.15.172:6789/0 conn(0x7f2094063230 legacy=0x7f209406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f2098fef700 1 -- 172.21.15.7:0/1095873434 >> v1:172.21.15.7:6789/0 conn(0x7f20940c9ec0 legacy=0x7f2094158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f2098fef700 1 -- 172.21.15.7:0/1095873434 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209415a8f0 con 0x7f20940c9af0 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f2098fef700 1 --2- 172.21.15.7:0/1095873434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20940c9af0 0x7f209415d8f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2084009910 tx=0x7f20840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4db0ad0ddda18656 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f20937fe700 1 -- 172.21.15.7:0/1095873434 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084011630 con 0x7f20940c9af0 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.257+0000 7f20937fe700 1 -- 172.21.15.7:0/1095873434 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2084011790 con 0x7f20940c9af0 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.258+0000 7f20937fe700 1 -- 172.21.15.7:0/1095873434 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084011b30 con 0x7f20940c9af0 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.258+0000 7f209b253700 1 -- 172.21.15.7:0/1095873434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20940c9af0 msgr2=0x7f209415d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.258+0000 7f209b253700 1 --2- 172.21.15.7:0/1095873434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20940c9af0 0x7f209415d8f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2084009910 tx=0x7f20840091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.258+0000 7f209b253700 1 -- 172.21.15.7:0/1095873434 shutdown_connections 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.258+0000 7f209b253700 1 --2- 172.21.15.7:0/1095873434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20940c9af0 0x7f209415d8f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.258+0000 7f209b253700 1 -- 172.21.15.7:0/1095873434 >> 172.21.15.7:0/1095873434 conn(0x7f20940cd1a0 msgr2=0x7f20940c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.258+0000 7f209b253700 1 -- 172.21.15.7:0/1095873434 shutdown_connections 2023-10-07T07:58:28.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.258+0000 7f209b253700 1 -- 172.21.15.7:0/1095873434 wait complete. 2023-10-07T07:58:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.259+0000 7f209b253700 1 Processor -- start 2023-10-07T07:58:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.260+0000 7f209b253700 1 -- start start 2023-10-07T07:58:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.260+0000 7f209b253700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2094063230 0x7f20941551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.260+0000 7f209b253700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20940c9af0 0x7f209416a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.260+0000 7f209b253700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20940c9ec0 0x7f20940c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.260+0000 7f209b253700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20941556f0 con 0x7f20940c9ec0 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.260+0000 7f209b253700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20940a64a0 con 0x7f20940c9af0 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.260+0000 7f209b253700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209416a840 con 0x7f2094063230 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f20997f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20940c9ec0 0x7f20940c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f20997f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20940c9ec0 0x7f20940c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47088/0 (socket says 172.21.15.7:47088) 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f20997f0700 1 -- 172.21.15.7:0/1393535527 learned_addr learned my addr 172.21.15.7:0/1393535527 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f2093fff700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20940c9af0 0x7f209416a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f2098fef700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2094063230 0x7f20941551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f20997f0700 1 -- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2094063230 msgr2=0x7f20941551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f20997f0700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2094063230 0x7f20941551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f20997f0700 1 -- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20940c9af0 msgr2=0x7f209416a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f20997f0700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20940c9af0 0x7f209416a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f20997f0700 1 -- 172.21.15.7:0/1393535527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2084009420 con 0x7f20940c9ec0 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f2093fff700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20940c9af0 0x7f209416a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f2098fef700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2094063230 0x7f20941551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f20997f0700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20940c9ec0 0x7f20940c64a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f208c00cc90 tx=0x7f208c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f2090ff9700 1 -- 172.21.15.7:0/1393535527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00cea0 con 0x7f20940c9ec0 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f2090ff9700 1 -- 172.21.15.7:0/1393535527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c00ab30 con 0x7f20940c9ec0 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20940a61c0 con 0x7f20940c9ec0 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f2090ff9700 1 -- 172.21.15.7:0/1393535527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c0070f0 con 0x7f20940c9ec0 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.261+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20940c5ca0 con 0x7f20940c9ec0 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.262+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20940c8b60 con 0x7f20940c9ec0 2023-10-07T07:58:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.263+0000 7f2090ff9700 1 -- 172.21.15.7:0/1393535527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f208c0074e0 con 0x7f20940c9ec0 2023-10-07T07:58:28.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.263+0000 7f2090ff9700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2074066f30 0x7f20740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.263+0000 7f2090ff9700 1 -- 172.21.15.7:0/1393535527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f208c0849a0 con 0x7f20940c9ec0 2023-10-07T07:58:28.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.264+0000 7f2098fef700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2074066f30 0x7f20740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.264+0000 7f2098fef700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2074066f30 0x7f20740693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2084009860 tx=0x7f2084017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:28.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.268+0000 7f2090ff9700 1 -- 172.21.15.7:0/1393535527 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f208c051c90 con 0x7f20940c9ec0 2023-10-07T07:58:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.525+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f20940716c0 con 0x7f20940c9ec0 2023-10-07T07:58:28.526 INFO:tasks.ceph.mon.a.smithi007.stderr:2023-10-07T07:58:28.525+0000 7feeb1c30700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T07:58:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.529+0000 7f2090ff9700 1 -- 172.21.15.7:0/1393535527 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f208c0552b0 con 0x7f20940c9ec0 2023-10-07T07:58:28.532 INFO:tasks.ceph.mds.c.smithi007.stderr: -2228> 2023-10-07T07:57:53.625+0000 7fba013fd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T07:58:28.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2074066f30 msgr2=0x7f20740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2074066f30 0x7f20740693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2084009860 tx=0x7f2084017000 comp rx=0 tx=0).stop 2023-10-07T07:58:28.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20940c9ec0 msgr2=0x7f20940c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20940c9ec0 0x7f20940c64a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f208c00cc90 tx=0x7f208c001ae0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 shutdown_connections 2023-10-07T07:58:28.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2074066f30 0x7f20740693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20940c9ec0 0x7f20940c64a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2094063230 0x7f20941551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 --2- 172.21.15.7:0/1393535527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20940c9af0 0x7f209416a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 >> 172.21.15.7:0/1393535527 conn(0x7f20940cd1a0 msgr2=0x7f20941592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:28.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 shutdown_connections 2023-10-07T07:58:28.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.532+0000 7f209b253700 1 -- 172.21.15.7:0/1393535527 wait complete. 2023-10-07T07:58:28.534 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-07T07:58:28.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-07T07:58:28.558 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2023-10-07T07:58:28.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.680+0000 7f4cc13fd700 1 Processor -- start 2023-10-07T07:58:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.680+0000 7f4cc13fd700 1 -- start start 2023-10-07T07:58:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.681+0000 7f4cc13fd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4cbc0b39b0 0x7f4cbc068100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.681+0000 7f4cc13fd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.681+0000 7f4cc13fd700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cbc068640 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.681+0000 7f4cc13fd700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cbc0b1130 con 0x7f4cbc0b39b0 2023-10-07T07:58:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.681+0000 7f4cc13fd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc15a8e0 con 0x7f4cbc0b14a0 2023-10-07T07:58:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cba7fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cba7fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc0afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:49132/0 (socket says 172.21.15.7:49132) 2023-10-07T07:58:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cbaffd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4cbc0b39b0 0x7f4cbc068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cba7fc700 1 -- 172.21.15.7:0/2321269433 learned_addr learned my addr 172.21.15.7:0/2321269433 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:28.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cba7fc700 1 -- 172.21.15.7:0/2321269433 >> v2:172.21.15.172:3300/0 conn(0x7f4cbc0b39b0 msgr2=0x7f4cbc068100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:28.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cba7fc700 1 --2- 172.21.15.7:0/2321269433 >> v2:172.21.15.172:3300/0 conn(0x7f4cbc0b39b0 0x7f4cbc068100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cba7fc700 1 -- 172.21.15.7:0/2321269433 >> v1:172.21.15.172:6789/0 conn(0x7f4cbc0b18f0 legacy=0x7f4cbc15d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cb9ffb700 1 -- 172.21.15.7:0/2321269433 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919497700 0 0) 0x7f4cbc068640 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cba7fc700 1 -- 172.21.15.7:0/2321269433 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc159cd0 con 0x7f4cbc0b14a0 2023-10-07T07:58:28.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cba7fc700 1 --2- 172.21.15.7:0/2321269433 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc0afa80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0099b0 tx=0x7f4cac009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc9dfb9bfd6ef178 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:28.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.682+0000 7f4cb9ffb700 1 -- 172.21.15.7:0/2321269433 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac009ea0 con 0x7f4cbc0b14a0 2023-10-07T07:58:28.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cb9ffb700 1 -- 172.21.15.7:0/2321269433 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac00b860 con 0x7f4cbc0b14a0 2023-10-07T07:58:28.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cb9ffb700 1 -- 172.21.15.7:0/2321269433 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00bc00 con 0x7f4cbc0b14a0 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cc13fd700 1 -- 172.21.15.7:0/2321269433 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 msgr2=0x7f4cbc0afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cc13fd700 1 --2- 172.21.15.7:0/2321269433 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc0afa80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0099b0 tx=0x7f4cac009250 comp rx=0 tx=0).stop 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cc13fd700 1 -- 172.21.15.7:0/2321269433 shutdown_connections 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cc13fd700 1 --2- 172.21.15.7:0/2321269433 >> v2:172.21.15.172:3300/0 conn(0x7f4cbc0b39b0 0x7f4cbc068100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cc13fd700 1 --2- 172.21.15.7:0/2321269433 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc0afa80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cc13fd700 1 -- 172.21.15.7:0/2321269433 >> 172.21.15.7:0/2321269433 conn(0x7f4cbc06d350 msgr2=0x7f4cbc06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cc13fd700 1 -- 172.21.15.7:0/2321269433 shutdown_connections 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.683+0000 7f4cc13fd700 1 -- 172.21.15.7:0/2321269433 wait complete. 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.685+0000 7f4cc13fd700 1 Processor -- start 2023-10-07T07:58:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.685+0000 7f4cc13fd700 1 -- start start 2023-10-07T07:58:28.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.685+0000 7f4cc13fd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.685+0000 7f4cc13fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc0b18f0 0x7f4cbc0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.685+0000 7f4cc13fd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cbc0b39b0 0x7f4cbc0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.685+0000 7f4cc13fd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc14ece0 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.685+0000 7f4cc13fd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc0c8620 con 0x7f4cbc0b39b0 2023-10-07T07:58:28.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.685+0000 7f4cc13fd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc0cc2e0 con 0x7f4cbc0b14a0 2023-10-07T07:58:28.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.685+0000 7f4cba7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc0b18f0 0x7f4cbc0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cba7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc0b18f0 0x7f4cbc0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47098/0 (socket says 172.21.15.7:47098) 2023-10-07T07:58:28.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cba7fc700 1 -- 172.21.15.7:0/1147206900 learned_addr learned my addr 172.21.15.7:0/1147206900 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cbb7fe700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cbc0b39b0 0x7f4cbc0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cbaffd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cba7fc700 1 -- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 msgr2=0x7f4cbc14cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cba7fc700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc14cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cba7fc700 1 -- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cbc0b39b0 msgr2=0x7f4cbc0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cba7fc700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cbc0b39b0 0x7f4cbc0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cba7fc700 1 -- 172.21.15.7:0/1147206900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac0094f0 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cba7fc700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc0b18f0 0x7f4cbc0ccd40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc15aa20 tx=0x7f4cac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cb2ffd700 1 -- 172.21.15.7:0/1147206900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac000d10 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cb2ffd700 1 -- 172.21.15.7:0/1147206900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac00bbe0 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cbc0cb420 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cb2ffd700 1 -- 172.21.15.7:0/1147206900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac011690 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.686+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc0cb780 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.687+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cbc0cc450 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.688+0000 7f4cb2ffd700 1 -- 172.21.15.7:0/1147206900 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4cac011ac0 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.688+0000 7f4cb2ffd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c9c066f30 0x7f4c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:28.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.688+0000 7f4cb2ffd700 1 -- 172.21.15.7:0/1147206900 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f4cac085e70 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.688+0000 7f4cbaffd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c9c066f30 0x7f4c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:28.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.689+0000 7f4cbaffd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c9c066f30 0x7f4c9c0693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009d10 tx=0x7f4ca40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:28.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.693+0000 7f4cb2ffd700 1 -- 172.21.15.7:0/1147206900 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cac053090 con 0x7f4cbc0b18f0 2023-10-07T07:58:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:28.942+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4cbc159490 con 0x7f4cbc0b18f0 2023-10-07T07:58:29.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.530+0000 7f4cb2ffd700 1 -- 172.21.15.7:0/1147206900 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4cac0566b0 con 0x7f4cbc0b18f0 2023-10-07T07:58:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c9c066f30 msgr2=0x7f4c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c9c066f30 0x7f4c9c0693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009d10 tx=0x7f4ca40091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc0b18f0 msgr2=0x7f4cbc0ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc0b18f0 0x7f4cbc0ccd40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc15aa20 tx=0x7f4cac017000 comp rx=0 tx=0).stop 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 shutdown_connections 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c9c066f30 0x7f4c9c0693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cbc0b39b0 0x7f4cbc0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cbc0b14a0 0x7f4cbc14cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 --2- 172.21.15.7:0/1147206900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc0b18f0 0x7f4cbc0ccd40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 >> 172.21.15.7:0/1147206900 conn(0x7f4cbc06d350 msgr2=0x7f4cbc069680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 shutdown_connections 2023-10-07T07:58:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.533+0000 7f4cc13fd700 1 -- 172.21.15.7:0/1147206900 wait complete. 2023-10-07T07:58:29.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-07T07:58:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.733+0000 7f32c0129700 1 Processor -- start 2023-10-07T07:58:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.733+0000 7f32c0129700 1 -- start start 2023-10-07T07:58:29.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.734+0000 7f32c0129700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f32b8067e40 0x7f32b80b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:29.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.734+0000 7f32c0129700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b80cd730 con 0x7f32b8068210 2023-10-07T07:58:29.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.734+0000 7f32c0129700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b80b1aa0 con 0x7f32b815d950 2023-10-07T07:58:29.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.734+0000 7f32c0129700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b80b1d80 con 0x7f32b8067e40 2023-10-07T07:58:29.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.734+0000 7f32bd6c4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f32b8068210 0x7f32b815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55090/0 (socket says 172.21.15.7:55090) 2023-10-07T07:58:29.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.734+0000 7f32bd6c4700 1 -- 172.21.15.7:0/900616066 learned_addr learned my addr 172.21.15.7:0/900616066 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:29.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.734+0000 7f32bdec5700 1 --2- 172.21.15.7:0/900616066 >> v2:172.21.15.172:3300/0 conn(0x7f32b8067e40 0x7f32b80b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.734+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114006043 0 0) 0x7f32b80cd730 con 0x7f32b8068210 2023-10-07T07:58:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.734+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a4003570 con 0x7f32b8068210 2023-10-07T07:58:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2342708217 0 0) 0x7f32a4003570 con 0x7f32b8068210 2023-10-07T07:58:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 >> v2:172.21.15.172:3300/0 conn(0x7f32b8067e40 msgr2=0x7f32b80b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 --2- 172.21.15.7:0/900616066 >> v2:172.21.15.172:3300/0 conn(0x7f32b8067e40 0x7f32b80b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 >> v1:172.21.15.172:6789/0 conn(0x7f32b815d950 legacy=0x7f32b80cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80b2060 con 0x7f32b8068210 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f32a8001470 con 0x7f32b8068210 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 >> v1:172.21.15.7:6789/0 conn(0x7f32b8068210 legacy=0x7f32b815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32a4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a4004810 0x7f32a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32a4005210 0x7f32a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80b1d80 con 0x7f32a4003ab0 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a400c130 con 0x7f32a4004810 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a400c2f0 con 0x7f32a4005210 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bdec5700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32a4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bd6c4700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a4004810 0x7f32a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32be6c6700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32a4005210 0x7f32a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bdec5700 1 -- 172.21.15.7:0/900616066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32a4005210 msgr2=0x7f32a400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bdec5700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32a4005210 0x7f32a400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bdec5700 1 -- 172.21.15.7:0/900616066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a4004810 msgr2=0x7f32a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bdec5700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a4004810 0x7f32a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bdec5700 1 -- 172.21.15.7:0/900616066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4003570 con 0x7f32a4003ab0 2023-10-07T07:58:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32be6c6700 1 -- 172.21.15.7:0/900616066 reap_dead start 2023-10-07T07:58:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.735+0000 7f32bdec5700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32a4003ec0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009c00 tx=0x7f32b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564cfe5fd86e06df server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400e710 con 0x7f32a4003ab0 2023-10-07T07:58:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b400e870 con 0x7f32a4003ab0 2023-10-07T07:58:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32bcec3700 1 -- 172.21.15.7:0/900616066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400ec30 con 0x7f32a4003ab0 2023-10-07T07:58:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32c0129700 1 -- 172.21.15.7:0/900616066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 msgr2=0x7f32a4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32c0129700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32a4003ec0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009c00 tx=0x7f32b40091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32c0129700 1 -- 172.21.15.7:0/900616066 shutdown_connections 2023-10-07T07:58:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32c0129700 1 --2- 172.21.15.7:0/900616066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32a4003ec0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32c0129700 1 -- 172.21.15.7:0/900616066 >> 172.21.15.7:0/900616066 conn(0x7f32b8069e90 msgr2=0x7f32b806e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32c0129700 1 -- 172.21.15.7:0/900616066 shutdown_connections 2023-10-07T07:58:29.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.736+0000 7f32c0129700 1 -- 172.21.15.7:0/900616066 wait complete. 2023-10-07T07:58:29.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.738+0000 7f32c0129700 1 Processor -- start 2023-10-07T07:58:29.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.738+0000 7f32c0129700 1 -- start start 2023-10-07T07:58:29.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32c0129700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32b80cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:29.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32c0129700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80ce3e0 0x7f32b816f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:29.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32c0129700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32b816df10 0x7f32b80c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:29.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32c0129700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8151fd0 con 0x7f32a4003ab0 2023-10-07T07:58:29.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32c0129700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80cffe0 con 0x7f32b80ce3e0 2023-10-07T07:58:29.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32c0129700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80ced70 con 0x7f32b816df10 2023-10-07T07:58:29.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bdec5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32b80cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:29.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bdec5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32b80cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34004/0 (socket says 172.21.15.7:34004) 2023-10-07T07:58:29.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bdec5700 1 -- 172.21.15.7:0/4226995123 learned_addr learned my addr 172.21.15.7:0/4226995123 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:29.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32be6c6700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32b816df10 0x7f32b80c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:29.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bd6c4700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80ce3e0 0x7f32b816f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:29.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bdec5700 1 -- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32b816df10 msgr2=0x7f32b80c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:29.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bdec5700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32b816df10 0x7f32b80c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bdec5700 1 -- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80ce3e0 msgr2=0x7f32b816f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:29.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bdec5700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80ce3e0 0x7f32b816f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:29.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bdec5700 1 -- 172.21.15.7:0/4226995123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80ce9a0 con 0x7f32a4003ab0 2023-10-07T07:58:29.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32bdec5700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32b80cfaa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e4b0 tx=0x7f32b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:29.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32adffb700 1 -- 172.21.15.7:0/4226995123 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4022070 con 0x7f32a4003ab0 2023-10-07T07:58:29.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32adffb700 1 -- 172.21.15.7:0/4226995123 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b400e9e0 con 0x7f32a4003ab0 2023-10-07T07:58:29.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32adffb700 1 -- 172.21.15.7:0/4226995123 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4027470 con 0x7f32a4003ab0 2023-10-07T07:58:29.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.739+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b80cefc0 con 0x7f32a4003ab0 2023-10-07T07:58:29.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.740+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b80cf580 con 0x7f32a4003ab0 2023-10-07T07:58:29.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.741+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b80b0c90 con 0x7f32a4003ab0 2023-10-07T07:58:29.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.741+0000 7f32adffb700 1 -- 172.21.15.7:0/4226995123 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32b401c450 con 0x7f32a4003ab0 2023-10-07T07:58:29.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.741+0000 7f32adffb700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f329c066f30 0x7f329c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:29.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.742+0000 7f32adffb700 1 -- 172.21.15.7:0/4226995123 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f32b40860e0 con 0x7f32a4003ab0 2023-10-07T07:58:29.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.742+0000 7f32bd6c4700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f329c066f30 0x7f329c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:29.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.742+0000 7f32bd6c4700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f329c066f30 0x7f329c0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f32b8161b90 tx=0x7f32a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:29.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.746+0000 7f32adffb700 1 -- 172.21.15.7:0/4226995123 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32b4053300 con 0x7f32a4003ab0 2023-10-07T07:58:29.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:29.948+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32b80716c0 con 0x7f32a4003ab0 2023-10-07T07:58:30.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.539+0000 7f32adffb700 1 -- 172.21.15.7:0/4226995123 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f32b4006cb0 con 0x7f32a4003ab0 2023-10-07T07:58:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.542+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f329c066f30 msgr2=0x7f329c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.542+0000 7f32c0129700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f329c066f30 0x7f329c0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f32b8161b90 tx=0x7f32a8009040 comp rx=0 tx=0).stop 2023-10-07T07:58:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.542+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 msgr2=0x7f32b80cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.542+0000 7f32c0129700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32b80cfaa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e4b0 tx=0x7f32b401a000 comp rx=0 tx=0).stop 2023-10-07T07:58:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.542+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 shutdown_connections 2023-10-07T07:58:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.543+0000 7f32c0129700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f329c066f30 0x7f329c0693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.543+0000 7f32c0129700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32b816df10 0x7f32b80c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.543+0000 7f32c0129700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a4003ab0 0x7f32b80cfaa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:30.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.543+0000 7f32c0129700 1 --2- 172.21.15.7:0/4226995123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80ce3e0 0x7f32b816f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:30.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.543+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 >> 172.21.15.7:0/4226995123 conn(0x7f32b8069e90 msgr2=0x7f32b80b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:30.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.543+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 shutdown_connections 2023-10-07T07:58:30.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.543+0000 7f32c0129700 1 -- 172.21.15.7:0/4226995123 wait complete. 2023-10-07T07:58:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2023-10-07T07:58:30.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-07T07:58:30.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.701+0000 7fa221c8c700 1 Processor -- start 2023-10-07T07:58:30.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.701+0000 7fa221c8c700 1 -- start start 2023-10-07T07:58:30.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.702+0000 7fa221c8c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa21c1594f0 0x7fa21c06af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:30.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.702+0000 7fa221c8c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa21c0c1d30 0x7fa21c0c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.702+0000 7fa221c8c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21c0c26d0 con 0x7fa21c1598c0 2023-10-07T07:58:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.702+0000 7fa221c8c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c0a9920 con 0x7fa21c0c1d30 2023-10-07T07:58:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.702+0000 7fa221c8c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c06b4f0 con 0x7fa21c1594f0 2023-10-07T07:58:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.702+0000 7fa21bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa21c0c1d30 0x7fa21c0c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.702+0000 7fa21affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa21c1598c0 0x7fa21c0674f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55100/0 (socket says 172.21.15.7:55100) 2023-10-07T07:58:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.702+0000 7fa21affd700 1 -- 172.21.15.7:0/3656785263 learned_addr learned my addr 172.21.15.7:0/3656785263 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21b7fe700 1 --2- 172.21.15.7:0/3656785263 >> v2:172.21.15.172:3300/0 conn(0x7fa21c1594f0 0x7fa21c06af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21bfff700 1 -- 172.21.15.7:0/3656785263 >> v2:172.21.15.172:3300/0 conn(0x7fa21c1594f0 msgr2=0x7fa21c06af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21bfff700 1 --2- 172.21.15.7:0/3656785263 >> v2:172.21.15.172:3300/0 conn(0x7fa21c1594f0 0x7fa21c06af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21bfff700 1 -- 172.21.15.7:0/3656785263 >> v1:172.21.15.7:6789/0 conn(0x7fa21c1598c0 legacy=0x7fa21c0674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21bfff700 1 -- 172.21.15.7:0/3656785263 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c06b7d0 con 0x7fa21c0c1d30 2023-10-07T07:58:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21b7fe700 1 --2- 172.21.15.7:0/3656785263 >> v2:172.21.15.172:3300/0 conn(0x7fa21c1594f0 0x7fa21c06af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21a7fc700 1 -- 172.21.15.7:0/3656785263 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741816803 0 0) 0x7fa210001110 con 0x7fa21c1598c0 2023-10-07T07:58:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21bfff700 1 --2- 172.21.15.7:0/3656785263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c0c1d30 0x7fa21c0c2190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0ccea0 tx=0x7fa214009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f366e9a808f6ed server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21a7fc700 1 -- 172.21.15.7:0/3656785263 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400e070 con 0x7fa21c0c1d30 2023-10-07T07:58:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21a7fc700 1 -- 172.21.15.7:0/3656785263 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21400bc30 con 0x7fa21c0c1d30 2023-10-07T07:58:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.703+0000 7fa21a7fc700 1 -- 172.21.15.7:0/3656785263 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214012430 con 0x7fa21c0c1d30 2023-10-07T07:58:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.704+0000 7fa221c8c700 1 -- 172.21.15.7:0/3656785263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c0c1d30 msgr2=0x7fa21c0c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:30.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.704+0000 7fa221c8c700 1 --2- 172.21.15.7:0/3656785263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c0c1d30 0x7fa21c0c2190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0ccea0 tx=0x7fa214009780 comp rx=0 tx=0).stop 2023-10-07T07:58:30.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.704+0000 7fa221c8c700 1 -- 172.21.15.7:0/3656785263 shutdown_connections 2023-10-07T07:58:30.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.704+0000 7fa221c8c700 1 --2- 172.21.15.7:0/3656785263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c0c1d30 0x7fa21c0c2190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:30.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.704+0000 7fa221c8c700 1 --2- 172.21.15.7:0/3656785263 >> v2:172.21.15.172:3300/0 conn(0x7fa21c1594f0 0x7fa21c06af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:30.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.704+0000 7fa221c8c700 1 -- 172.21.15.7:0/3656785263 >> 172.21.15.7:0/3656785263 conn(0x7fa21c0cda60 msgr2=0x7fa21c068100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:30.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.704+0000 7fa221c8c700 1 -- 172.21.15.7:0/3656785263 shutdown_connections 2023-10-07T07:58:30.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.704+0000 7fa221c8c700 1 -- 172.21.15.7:0/3656785263 wait complete. 2023-10-07T07:58:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.705+0000 7fa221c8c700 1 Processor -- start 2023-10-07T07:58:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.705+0000 7fa221c8c700 1 -- start start 2023-10-07T07:58:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa221c8c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa21c0c1d30 0x7fa21c169a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa221c8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c1594f0 0x7fa21c15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa221c8c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa21c1598c0 0x7fa21c0c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa221c8c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0c69a0 con 0x7fa21c1594f0 2023-10-07T07:58:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa221c8c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0c6510 con 0x7fa21c1598c0 2023-10-07T07:58:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa221c8c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c169fd0 con 0x7fa21c0c1d30 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c1594f0 0x7fa21c15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c1594f0 0x7fa21c15b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34032/0 (socket says 172.21.15.7:34032) 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21affd700 1 -- 172.21.15.7:0/3774157771 learned_addr learned my addr 172.21.15.7:0/3774157771 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21bfff700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa21c1598c0 0x7fa21c0c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21b7fe700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa21c0c1d30 0x7fa21c169a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21affd700 1 -- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa21c0c1d30 msgr2=0x7fa21c169a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21affd700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa21c0c1d30 0x7fa21c169a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21affd700 1 -- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa21c1598c0 msgr2=0x7fa21c0c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21affd700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa21c1598c0 0x7fa21c0c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21affd700 1 -- 172.21.15.7:0/3774157771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa214009a20 con 0x7fa21c1594f0 2023-10-07T07:58:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.706+0000 7fa21bfff700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa21c1598c0 0x7fa21c0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.707+0000 7fa21affd700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c1594f0 0x7fa21c15b780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa21000c5b0 tx=0x7fa210001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.707+0000 7fa2037fe700 1 -- 172.21.15.7:0/3774157771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210005b30 con 0x7fa21c1594f0 2023-10-07T07:58:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.707+0000 7fa2037fe700 1 -- 172.21.15.7:0/3774157771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa210005c90 con 0x7fa21c1594f0 2023-10-07T07:58:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.707+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c0c5900 con 0x7fa21c1594f0 2023-10-07T07:58:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.707+0000 7fa2037fe700 1 -- 172.21.15.7:0/3774157771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210007300 con 0x7fa21c1594f0 2023-10-07T07:58:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.707+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c0c5ec0 con 0x7fa21c1594f0 2023-10-07T07:58:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.708+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c0c6680 con 0x7fa21c1594f0 2023-10-07T07:58:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.708+0000 7fa2037fe700 1 -- 172.21.15.7:0/3774157771 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2100076b0 con 0x7fa21c1594f0 2023-10-07T07:58:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.709+0000 7fa2037fe700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa208066f30 0x7fa2080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.709+0000 7fa2037fe700 1 -- 172.21.15.7:0/3774157771 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fa210083780 con 0x7fa21c1594f0 2023-10-07T07:58:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.709+0000 7fa21b7fe700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa208066f30 0x7fa2080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.710+0000 7fa21b7fe700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa208066f30 0x7fa2080693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa21c15c850 tx=0x7fa20c006f90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.713+0000 7fa2037fe700 1 -- 172.21.15.7:0/3774157771 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa210050ab0 con 0x7fa21c1594f0 2023-10-07T07:58:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:30.926+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa21c0c26d0 con 0x7fa21c1594f0 2023-10-07T07:58:31.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.541+0000 7fa2037fe700 1 -- 172.21.15.7:0/3774157771 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa21c0c26d0 con 0x7fa21c1594f0 2023-10-07T07:58:31.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa208066f30 msgr2=0x7fa2080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:31.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa208066f30 0x7fa2080693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa21c15c850 tx=0x7fa20c006f90 comp rx=0 tx=0).stop 2023-10-07T07:58:31.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c1594f0 msgr2=0x7fa21c15b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:31.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c1594f0 0x7fa21c15b780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa21000c5b0 tx=0x7fa210001070 comp rx=0 tx=0).stop 2023-10-07T07:58:31.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 shutdown_connections 2023-10-07T07:58:31.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa208066f30 0x7fa2080693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:31.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa21c1598c0 0x7fa21c0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:31.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa21c0c1d30 0x7fa21c169a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:31.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 --2- 172.21.15.7:0/3774157771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa21c1594f0 0x7fa21c15b780 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:31.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 >> 172.21.15.7:0/3774157771 conn(0x7fa21c0cda60 msgr2=0x7fa21c157e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:31.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 shutdown_connections 2023-10-07T07:58:31.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.544+0000 7fa221c8c700 1 -- 172.21.15.7:0/3774157771 wait complete. 2023-10-07T07:58:31.546 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2023-10-07T07:58:31.565 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-07T07:58:31.565 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-07T07:58:31.565 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-07T07:58:31.565 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T07:58:31.565 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-07T07:58:31.566 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-07T07:58:31.566 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-07T07:58:31.566 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T07:58:31.566 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-07T07:58:31.566 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-07T07:58:31.566 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-07T07:58:31.566 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T07:58:31.567 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-07T07:58:31.567 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-07T07:58:31.567 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-07T07:58:31.567 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T07:58:31.576 INFO:tasks.ceph.mds.a:Stopped 2023-10-07T07:58:31.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-07T07:58:31.577 INFO:tasks.ceph.mds.c:Stopped 2023-10-07T07:58:31.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-07T07:58:31.578 INFO:tasks.ceph.mds.b:Stopped 2023-10-07T07:58:31.579 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-07T07:58:31.579 INFO:tasks.ceph.mds.d:Stopped 2023-10-07T07:58:31.579 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-07T07:58:31.619 INFO:tasks.ceph.mds.a:Started 2023-10-07T07:58:31.620 DEBUG:teuthology.parallel:result is None 2023-10-07T07:58:31.621 INFO:tasks.ceph.mds.c:Started 2023-10-07T07:58:31.621 DEBUG:teuthology.parallel:result is None 2023-10-07T07:58:31.622 INFO:tasks.ceph.mds.b:Started 2023-10-07T07:58:31.622 DEBUG:teuthology.parallel:result is None 2023-10-07T07:58:31.622 INFO:tasks.ceph.mds.d:Started 2023-10-07T07:58:31.622 DEBUG:teuthology.parallel:result is None 2023-10-07T07:58:31.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-07T07:58:31.703 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2023-10-07T07:58:31.704 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2023-10-07T07:58:31.712 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2023-10-07T07:58:31.712 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2023-10-07T07:58:31.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.767+0000 7f2745129700 1 Processor -- start 2023-10-07T07:58:31.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.767+0000 7f2745129700 1 -- start start 2023-10-07T07:58:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.768+0000 7f2745129700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2740063230 0x7f274006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.768+0000 7f2745129700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f27400c9ec0 0x7f274006e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.768+0000 7f2745129700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274006e590 con 0x7f27400c9af0 2023-10-07T07:58:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.768+0000 7f2745129700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27400b03a0 con 0x7f27400c9ec0 2023-10-07T07:58:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.768+0000 7f2745129700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274015a330 con 0x7f2740063230 2023-10-07T07:58:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273e59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2740063230 0x7f274006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273f59e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f27400c9ec0 0x7f274006e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273e59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2740063230 0x7f274006db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:48852/0 (socket says 172.21.15.7:48852) 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273e59c700 1 -- 172.21.15.7:0/3066023700 learned_addr learned my addr 172.21.15.7:0/3066023700 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273f59e700 1 -- 172.21.15.7:0/3066023700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2740063230 msgr2=0x7f274006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273f59e700 1 --2- 172.21.15.7:0/3066023700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2740063230 0x7f274006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273f59e700 1 -- 172.21.15.7:0/3066023700 >> v1:172.21.15.172:6789/0 conn(0x7f27400c9af0 legacy=0x7f274015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273f59e700 1 -- 172.21.15.7:0/3066023700 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274015a610 con 0x7f27400c9ec0 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273dd9b700 1 -- 172.21.15.7:0/3066023700 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842521320 0 0) 0x7f274006e590 con 0x7f27400c9af0 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273f59e700 1 --2- 172.21.15.7:0/3066023700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9ec0 0x7f274006e050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2738009e60 tx=0x7f2738009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77fed856189770a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273dd9b700 1 -- 172.21.15.7:0/3066023700 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800e070 con 0x7f27400c9ec0 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273dd9b700 1 -- 172.21.15.7:0/3066023700 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273800bb60 con 0x7f27400c9ec0 2023-10-07T07:58:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.769+0000 7f273dd9b700 1 -- 172.21.15.7:0/3066023700 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738012430 con 0x7f27400c9ec0 2023-10-07T07:58:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.770+0000 7f2745129700 1 -- 172.21.15.7:0/3066023700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9ec0 msgr2=0x7f274006e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.770+0000 7f2745129700 1 --2- 172.21.15.7:0/3066023700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9ec0 0x7f274006e050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2738009e60 tx=0x7f2738009780 comp rx=0 tx=0).stop 2023-10-07T07:58:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.770+0000 7f2745129700 1 -- 172.21.15.7:0/3066023700 shutdown_connections 2023-10-07T07:58:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.770+0000 7f2745129700 1 --2- 172.21.15.7:0/3066023700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9ec0 0x7f274006e050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.770+0000 7f2745129700 1 --2- 172.21.15.7:0/3066023700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2740063230 0x7f274006db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.770+0000 7f2745129700 1 -- 172.21.15.7:0/3066023700 >> 172.21.15.7:0/3066023700 conn(0x7f27400cd1a0 msgr2=0x7f27400c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.770+0000 7f2745129700 1 -- 172.21.15.7:0/3066023700 shutdown_connections 2023-10-07T07:58:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.770+0000 7f2745129700 1 -- 172.21.15.7:0/3066023700 wait complete. 2023-10-07T07:58:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.772+0000 7f2745129700 1 Processor -- start 2023-10-07T07:58:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.772+0000 7f2745129700 1 -- start start 2023-10-07T07:58:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f2745129700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2740063230 0x7f27400cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f2745129700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9af0 0x7f27401543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f2745129700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27400c9ec0 0x7f27400c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f2745129700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274015ce30 con 0x7f2740063230 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f2745129700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27400ca620 con 0x7f27400c9af0 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f2745129700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27400ca900 con 0x7f27400c9ec0 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2740063230 0x7f27400cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2740063230 0x7f27400cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34050/0 (socket says 172.21.15.7:34050) 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273ed9d700 1 -- 172.21.15.7:0/909895325 learned_addr learned my addr 172.21.15.7:0/909895325 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273e59c700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9af0 0x7f27401543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273f59e700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27400c9ec0 0x7f27400c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273ed9d700 1 -- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27400c9ec0 msgr2=0x7f27400c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273ed9d700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27400c9ec0 0x7f27400c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273ed9d700 1 -- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9af0 msgr2=0x7f27401543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273ed9d700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9af0 0x7f27401543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273ed9d700 1 -- 172.21.15.7:0/909895325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738009a20 con 0x7f2740063230 2023-10-07T07:58:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273e59c700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9af0 0x7f27401543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.773+0000 7f273ed9d700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2740063230 0x7f27400cb9a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f273000c7e0 tx=0x7f2730007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.774+0000 7f2726ffd700 1 -- 172.21.15.7:0/909895325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27300105c0 con 0x7f2740063230 2023-10-07T07:58:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.774+0000 7f2726ffd700 1 -- 172.21.15.7:0/909895325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2730010740 con 0x7f2740063230 2023-10-07T07:58:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.774+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27400cabe0 con 0x7f2740063230 2023-10-07T07:58:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.774+0000 7f2726ffd700 1 -- 172.21.15.7:0/909895325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2730010b00 con 0x7f2740063230 2023-10-07T07:58:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.774+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27400cb1a0 con 0x7f2740063230 2023-10-07T07:58:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.775+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27400716c0 con 0x7f2740063230 2023-10-07T07:58:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.775+0000 7f2726ffd700 1 -- 172.21.15.7:0/909895325 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f273001b3b0 con 0x7f2740063230 2023-10-07T07:58:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.776+0000 7f2726ffd700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f272c066f30 0x7f272c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.776+0000 7f2726ffd700 1 -- 172.21.15.7:0/909895325 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2730083820 con 0x7f2740063230 2023-10-07T07:58:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.776+0000 7f273e59c700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f272c066f30 0x7f272c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.777+0000 7f273e59c700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f272c066f30 0x7f272c0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27400675f0 tx=0x7f2734009250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.780+0000 7f2726ffd700 1 -- 172.21.15.7:0/909895325 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2730050d90 con 0x7f2740063230 2023-10-07T07:58:31.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:31.997+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2740158890 con 0x7f2740063230 2023-10-07T07:58:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.255+0000 7f2726ffd700 1 -- 172.21.15.7:0/909895325 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2730017070 con 0x7f2740063230 2023-10-07T07:58:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f272c066f30 msgr2=0x7f272c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f272c066f30 0x7f272c0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27400675f0 tx=0x7f2734009250 comp rx=0 tx=0).stop 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2740063230 msgr2=0x7f27400cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2740063230 0x7f27400cb9a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f273000c7e0 tx=0x7f2730007df0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 shutdown_connections 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f272c066f30 0x7f272c0693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27400c9ec0 0x7f27400c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2740063230 0x7f27400cb9a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 --2- 172.21.15.7:0/909895325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27400c9af0 0x7f27401543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 >> 172.21.15.7:0/909895325 conn(0x7f27400cd1a0 msgr2=0x7f2740069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 shutdown_connections 2023-10-07T07:58:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.258+0000 7f2745129700 1 -- 172.21.15.7:0/909895325 wait complete. 2023-10-07T07:58:33.261 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2023-10-07T07:58:33.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-07T07:58:33.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.410+0000 7fabf936f700 1 Processor -- start 2023-10-07T07:58:33.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.410+0000 7fabf936f700 1 -- start start 2023-10-07T07:58:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.411+0000 7fabf936f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabf40d0250 0x7fabf40d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.411+0000 7fabf936f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fabf40b3b60 0x7fabf40b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.411+0000 7fabf936f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf40b61f0 con 0x7fabf4159380 2023-10-07T07:58:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.411+0000 7fabf936f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf40d0be0 con 0x7fabf40d0250 2023-10-07T07:58:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.411+0000 7fabf936f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf40b4820 con 0x7fabf40b3b60 2023-10-07T07:58:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf27fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabf40d0250 0x7fabf40d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fabf4159380 0x7fabf40b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55132/0 (socket says 172.21.15.7:55132) 2023-10-07T07:58:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf2ffd700 1 -- 172.21.15.7:0/1161302385 learned_addr learned my addr 172.21.15.7:0/1161302385 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf37fe700 1 --2- 172.21.15.7:0/1161302385 >> v2:172.21.15.172:3300/0 conn(0x7fabf40b3b60 0x7fabf40b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf1ffb700 1 -- 172.21.15.7:0/1161302385 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582498598 0 0) 0x7fabf40b61f0 con 0x7fabf4159380 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf1ffb700 1 -- 172.21.15.7:0/1161302385 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe0003570 con 0x7fabf4159380 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf27fc700 1 -- 172.21.15.7:0/1161302385 >> v2:172.21.15.172:3300/0 conn(0x7fabf40b3b60 msgr2=0x7fabf40b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf27fc700 1 --2- 172.21.15.7:0/1161302385 >> v2:172.21.15.172:3300/0 conn(0x7fabf40b3b60 0x7fabf40b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf27fc700 1 -- 172.21.15.7:0/1161302385 >> v1:172.21.15.7:6789/0 conn(0x7fabf4159380 legacy=0x7fabf40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf27fc700 1 -- 172.21.15.7:0/1161302385 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf40b5000 con 0x7fabf40d0250 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf37fe700 1 --2- 172.21.15.7:0/1161302385 >> v2:172.21.15.172:3300/0 conn(0x7fabf40b3b60 0x7fabf40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf27fc700 1 --2- 172.21.15.7:0/1161302385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40d0250 0x7fabf40d06a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fabf40af100 tx=0x7fabdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88702c80d734f623 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf1ffb700 1 -- 172.21.15.7:0/1161302385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc011630 con 0x7fabf40d0250 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf1ffb700 1 -- 172.21.15.7:0/1161302385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc011790 con 0x7fabf40d0250 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.412+0000 7fabf1ffb700 1 -- 172.21.15.7:0/1161302385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc011b30 con 0x7fabf40d0250 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.413+0000 7fabf936f700 1 -- 172.21.15.7:0/1161302385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40d0250 msgr2=0x7fabf40d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.413+0000 7fabf936f700 1 --2- 172.21.15.7:0/1161302385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40d0250 0x7fabf40d06a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fabf40af100 tx=0x7fabdc0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.413+0000 7fabf936f700 1 -- 172.21.15.7:0/1161302385 shutdown_connections 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.413+0000 7fabf936f700 1 --2- 172.21.15.7:0/1161302385 >> v2:172.21.15.172:3300/0 conn(0x7fabf40b3b60 0x7fabf40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.413+0000 7fabf936f700 1 --2- 172.21.15.7:0/1161302385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40d0250 0x7fabf40d06a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.413+0000 7fabf936f700 1 -- 172.21.15.7:0/1161302385 >> 172.21.15.7:0/1161302385 conn(0x7fabf4067be0 msgr2=0x7fabf4067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.413+0000 7fabf936f700 1 -- 172.21.15.7:0/1161302385 shutdown_connections 2023-10-07T07:58:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.413+0000 7fabf936f700 1 -- 172.21.15.7:0/1161302385 wait complete. 2023-10-07T07:58:33.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.414+0000 7fabf936f700 1 Processor -- start 2023-10-07T07:58:33.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.415+0000 7fabf936f700 1 -- start start 2023-10-07T07:58:33.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.415+0000 7fabf936f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabf40b3b60 0x7fabf41692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.415+0000 7fabf936f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf40d0250 0x7fabf40b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.415+0000 7fabf936f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4159380 0x7fabf415bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.415+0000 7fabf936f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf416a4b0 con 0x7fabf4159380 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.415+0000 7fabf936f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4169b80 con 0x7fabf40d0250 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.415+0000 7fabf936f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4169e60 con 0x7fabf40b3b60 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.415+0000 7fabf37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4159380 0x7fabf415bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4159380 0x7fabf415bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34080/0 (socket says 172.21.15.7:34080) 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf37fe700 1 -- 172.21.15.7:0/3209517964 learned_addr learned my addr 172.21.15.7:0/3209517964 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf27fc700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf40d0250 0x7fabf40b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf2ffd700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabf40b3b60 0x7fabf41692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf37fe700 1 -- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabf40b3b60 msgr2=0x7fabf41692e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf37fe700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabf40b3b60 0x7fabf41692e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf37fe700 1 -- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf40d0250 msgr2=0x7fabf40b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf37fe700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf40d0250 0x7fabf40b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf37fe700 1 -- 172.21.15.7:0/3209517964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc009420 con 0x7fabf4159380 2023-10-07T07:58:33.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf37fe700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4159380 0x7fabf415bb70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabec00cef0 tx=0x7fabec007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:33.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabeaffd700 1 -- 172.21.15.7:0/3209517964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00ad20 con 0x7fabf4159380 2023-10-07T07:58:33.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabeaffd700 1 -- 172.21.15.7:0/3209517964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec016420 con 0x7fabf4159380 2023-10-07T07:58:33.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf416a140 con 0x7fabf4159380 2023-10-07T07:58:33.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabeaffd700 1 -- 172.21.15.7:0/3209517964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00aea0 con 0x7fabf4159380 2023-10-07T07:58:33.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.416+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf4169820 con 0x7fabf4159380 2023-10-07T07:58:33.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.417+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf416a940 con 0x7fabf4159380 2023-10-07T07:58:33.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.418+0000 7fabeaffd700 1 -- 172.21.15.7:0/3209517964 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabec016a70 con 0x7fabf4159380 2023-10-07T07:58:33.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.418+0000 7fabeaffd700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd4066fb0 0x7fabd4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.418+0000 7fabeaffd700 1 -- 172.21.15.7:0/3209517964 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fabec012070 con 0x7fabf4159380 2023-10-07T07:58:33.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.418+0000 7fabf2ffd700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd4066fb0 0x7fabd4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.419+0000 7fabf2ffd700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd4066fb0 0x7fabd4069460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007fd0 tx=0x7fabe4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:33.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.422+0000 7fabeaffd700 1 -- 172.21.15.7:0/3209517964 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabec04e5a0 con 0x7fabf4159380 2023-10-07T07:58:33.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.704+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fabf40716c0 con 0x7fabf4159380 2023-10-07T07:58:33.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.705+0000 7fabeaffd700 1 -- 172.21.15.7:0/3209517964 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fabf416a940 con 0x7fabf4159380 2023-10-07T07:58:33.706 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:33.706 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:58:33.706 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2023-10-07T07:58:33.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.706 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlCy6iHBAA0pyZcla7yG20KJxMwhhFVg==", 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.707 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlUxYOKhAAuXUwfiu6GTlHPau/HLhgJg==", 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.708 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFluHPbJBAAdTCLab+QEn7jhGF253MA4A==", 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlNaU3MRAAqpmLriqCxUsXuAyL5Cmbxw==", 2023-10-07T07:58:33.709 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBODyFlkgzdGRAAUXBLhY69mpInyEg/EZ1bbg==", 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:58:33.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBRDyFl95iAGBAA8mk47fzfwi2BDWqxoJIAGA==", 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBUDyFlpgWXFxAABhoYDIRsLPfY19Z76zlRfg==", 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBXDyFl9VThFhAAKbJJRsjjubjWK5xpt7j6Vg==", 2023-10-07T07:58:33.712 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBdDyFlcKXTGhAAPW0NNpqt2aoxJZUSdBtJEw==", 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:58:33.713 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBgDyFlYdmOGRAAO4UiW/o3zkFHKLo0A/Fl0g==", 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.715 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2023-10-07T07:58:33.715 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBjDyFlxgq2FxAAar/DNRzL+QMVm0sZJ6dY1A==", 2023-10-07T07:58:33.715 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:58:33.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:58:33.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.716 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2023-10-07T07:58:33.716 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBmDyFlZyNcFhAAfEAyrTMB6AkewTly3i3vVQ==", 2023-10-07T07:58:33.716 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:58:33.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:58:33.717 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.717 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2023-10-07T07:58:33.718 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlom77NRAA6mby9LFc/5XVOSXctSuffA==", 2023-10-07T07:58:33.718 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:58:33.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-07T07:58:33.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-07T07:58:33.718 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-07T07:58:33.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.719 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2023-10-07T07:58:33.720 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBNDyFlGLtzARAAh3o32z7sOz1RBqXINgae6g==", 2023-10-07T07:58:33.720 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:58:33.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-07T07:58:33.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-07T07:58:33.720 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-07T07:58:33.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.721 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2023-10-07T07:58:33.721 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlrCK0ORAAWpaeoXzMPo3ko8Yrat1bEg==", 2023-10-07T07:58:33.721 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:58:33.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-07T07:58:33.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-07T07:58:33.722 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-07T07:58:33.722 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.723 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2023-10-07T07:58:33.723 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBNDyFl/YjxBhAAivYmiSK5+fdwyFmyMeWI+A==", 2023-10-07T07:58:33.723 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:58:33.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-07T07:58:33.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-07T07:58:33.724 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-07T07:58:33.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.724 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2023-10-07T07:58:33.725 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBLDyFl3DSBOBAARv3edVYI1sfQLkk7iSk9IQ==", 2023-10-07T07:58:33.725 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2023-10-07T07:58:33.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:58:33.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:58:33.725 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.726 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mds", 2023-10-07T07:58:33.726 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBsDyFlZ00aHBAA8BSpEa7ECn97aq8cMd80VQ==", 2023-10-07T07:58:33.726 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mds" 2023-10-07T07:58:33.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.727 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mgr", 2023-10-07T07:58:33.727 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBsDyFl3VoaHBAA5IqUR2VyJp54qo+ZKBQPSQ==", 2023-10-07T07:58:33.728 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-07T07:58:33.728 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.728 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-osd", 2023-10-07T07:58:33.729 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBsDyFla2YaHBAAJ5U7ruzxHS2gFITobHOedA==", 2023-10-07T07:58:33.729 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-osd" 2023-10-07T07:58:33.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.730 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd", 2023-10-07T07:58:33.730 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBsDyFlAG8aHBAAtHGFkqAk4dW+MsZ6e8QEzw==", 2023-10-07T07:58:33.730 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-07T07:58:33.730 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.731 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-07T07:58:33.731 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBsDyFl+HcaHBAAlx00APiPNhcA56BtjA0IKw==", 2023-10-07T07:58:33.731 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-07T07:58:33.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.732 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rgw", 2023-10-07T07:58:33.732 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBsDyFlXoMaHBAAYcIs8peFUoIhtMpIhJu0lA==", 2023-10-07T07:58:33.732 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-07T07:58:33.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.733 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2023-10-07T07:58:33.733 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlgeSpGBAAlq+bBWoPBk7KPt5+xL1z4w==", 2023-10-07T07:58:33.733 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2023-10-07T07:58:33.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2023-10-07T07:58:33.734 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:33.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:33.735 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2023-10-07T07:58:33.735 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlhkIGFRAAWzKzlNAmIb3PXAlOaZMtYA==", 2023-10-07T07:58:33.735 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:58:33.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2023-10-07T07:58:33.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2023-10-07T07:58:33.735 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:58:33.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:33.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:58:33.736 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:58:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd4066fb0 msgr2=0x7fabd4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd4066fb0 0x7fabd4069460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007fd0 tx=0x7fabe4009040 comp rx=0 tx=0).stop 2023-10-07T07:58:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4159380 msgr2=0x7fabf415bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4159380 0x7fabf415bb70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabec00cef0 tx=0x7fabec007860 comp rx=0 tx=0).stop 2023-10-07T07:58:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 shutdown_connections 2023-10-07T07:58:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fabd4066fb0 0x7fabd4069460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4159380 0x7fabf415bb70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fabf40b3b60 0x7fabf41692e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 --2- 172.21.15.7:0/3209517964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf40d0250 0x7fabf40b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 >> 172.21.15.7:0/3209517964 conn(0x7fabf4067be0 msgr2=0x7fabf4157f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:33.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 shutdown_connections 2023-10-07T07:58:33.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.707+0000 7fabf936f700 1 -- 172.21.15.7:0/3209517964 wait complete. 2023-10-07T07:58:33.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-07T07:58:33.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.875+0000 7f86b1bd8700 1 Processor -- start 2023-10-07T07:58:33.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.875+0000 7f86b1bd8700 1 -- start start 2023-10-07T07:58:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.875+0000 7f86b1bd8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f86ac1533a0 0x7f86ac06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.875+0000 7f86b1bd8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86ac06d730 0x7f86ac159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86b1bd8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ac06ea20 con 0x7f86ac159960 2023-10-07T07:58:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86b1bd8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ac0c6880 con 0x7f86ac06d730 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86b1bd8700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ac159de0 con 0x7f86ac1533a0 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86ac06d730 0x7f86ac159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86abfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f86ac159960 0x7f86ac155e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55138/0 (socket says 172.21.15.7:55138) 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86abfff700 1 -- 172.21.15.7:0/77137524 learned_addr learned my addr 172.21.15.7:0/77137524 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86ab7fe700 1 --2- 172.21.15.7:0/77137524 >> v2:172.21.15.172:3300/0 conn(0x7f86ac1533a0 0x7f86ac06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aaffd700 1 -- 172.21.15.7:0/77137524 >> v2:172.21.15.172:3300/0 conn(0x7f86ac1533a0 msgr2=0x7f86ac06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aaffd700 1 --2- 172.21.15.7:0/77137524 >> v2:172.21.15.172:3300/0 conn(0x7f86ac1533a0 0x7f86ac06f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aaffd700 1 -- 172.21.15.7:0/77137524 >> v1:172.21.15.7:6789/0 conn(0x7f86ac159960 legacy=0x7f86ac155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aaffd700 1 -- 172.21.15.7:0/77137524 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac06dba0 con 0x7f86ac06d730 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aa7fc700 1 -- 172.21.15.7:0/77137524 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693891678 0 0) 0x7f86ac06ea20 con 0x7f86ac159960 2023-10-07T07:58:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aaffd700 1 --2- 172.21.15.7:0/77137524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac06d730 0x7f86ac159420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0c2ef0 tx=0x7f86a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2220f4ebbadbf4e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aa7fc700 1 -- 172.21.15.7:0/77137524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0009e80 con 0x7f86ac06d730 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aa7fc700 1 -- 172.21.15.7:0/77137524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a000b820 con 0x7f86ac06d730 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.876+0000 7f86aa7fc700 1 -- 172.21.15.7:0/77137524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a000bba0 con 0x7f86ac06d730 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.877+0000 7f86b1bd8700 1 -- 172.21.15.7:0/77137524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac06d730 msgr2=0x7f86ac159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.877+0000 7f86b1bd8700 1 --2- 172.21.15.7:0/77137524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac06d730 0x7f86ac159420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0c2ef0 tx=0x7f86a00091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.877+0000 7f86b1bd8700 1 -- 172.21.15.7:0/77137524 shutdown_connections 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.877+0000 7f86b1bd8700 1 --2- 172.21.15.7:0/77137524 >> v2:172.21.15.172:3300/0 conn(0x7f86ac1533a0 0x7f86ac06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.877+0000 7f86b1bd8700 1 --2- 172.21.15.7:0/77137524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac06d730 0x7f86ac159420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.877+0000 7f86b1bd8700 1 -- 172.21.15.7:0/77137524 >> 172.21.15.7:0/77137524 conn(0x7f86ac067a80 msgr2=0x7f86ac0cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.877+0000 7f86b1bd8700 1 -- 172.21.15.7:0/77137524 shutdown_connections 2023-10-07T07:58:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.877+0000 7f86b1bd8700 1 -- 172.21.15.7:0/77137524 wait complete. 2023-10-07T07:58:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.879+0000 7f86b1bd8700 1 Processor -- start 2023-10-07T07:58:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.879+0000 7f86b1bd8700 1 -- start start 2023-10-07T07:58:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.879+0000 7f86b1bd8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86ac06d730 0x7f86ac16a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.879+0000 7f86b1bd8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86ac1533a0 0x7f86ac16c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.879+0000 7f86b1bd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac159960 0x7f86ac0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.879+0000 7f86b1bd8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac16a5a0 con 0x7f86ac159960 2023-10-07T07:58:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.879+0000 7f86b1bd8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac0ac1c0 con 0x7f86ac06d730 2023-10-07T07:58:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.879+0000 7f86b1bd8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac0ab890 con 0x7f86ac1533a0 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac159960 0x7f86ac0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac159960 0x7f86ac0ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34110/0 (socket says 172.21.15.7:34110) 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86abfff700 1 -- 172.21.15.7:0/3214268543 learned_addr learned my addr 172.21.15.7:0/3214268543 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86ab7fe700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86ac06d730 0x7f86ac16a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86aaffd700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86ac1533a0 0x7f86ac16c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86abfff700 1 -- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86ac1533a0 msgr2=0x7f86ac16c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86abfff700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86ac1533a0 0x7f86ac16c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86abfff700 1 -- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86ac06d730 msgr2=0x7f86ac16a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86abfff700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86ac06d730 0x7f86ac16a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86abfff700 1 -- 172.21.15.7:0/3214268543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a0009420 con 0x7f86ac159960 2023-10-07T07:58:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86abfff700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac159960 0x7f86ac0ad8a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f86a400cc90 tx=0x7f86a4001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86937fe700 1 -- 172.21.15.7:0/3214268543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a400cea0 con 0x7f86ac159960 2023-10-07T07:58:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86937fe700 1 -- 172.21.15.7:0/3214268543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a400ab70 con 0x7f86ac159960 2023-10-07T07:58:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ac0abb70 con 0x7f86ac159960 2023-10-07T07:58:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86937fe700 1 -- 172.21.15.7:0/3214268543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a40070f0 con 0x7f86ac159960 2023-10-07T07:58:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.880+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ac16ac70 con 0x7f86ac159960 2023-10-07T07:58:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.881+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ac0ac350 con 0x7f86ac159960 2023-10-07T07:58:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.882+0000 7f86937fe700 1 -- 172.21.15.7:0/3214268543 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86a40074a0 con 0x7f86ac159960 2023-10-07T07:58:33.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.882+0000 7f86937fe700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8698066f30 0x7f86980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:33.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.882+0000 7f86937fe700 1 -- 172.21.15.7:0/3214268543 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f86a40846c0 con 0x7f86ac159960 2023-10-07T07:58:33.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.883+0000 7f86ab7fe700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8698066f30 0x7f86980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:33.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.883+0000 7f86ab7fe700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8698066f30 0x7f86980693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f869c006fd0 tx=0x7f869c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:33.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:33.887+0000 7f86937fe700 1 -- 172.21.15.7:0/3214268543 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86a4051d00 con 0x7f86ac159960 2023-10-07T07:58:34.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.169+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f86ac0716c0 con 0x7f86ac159960 2023-10-07T07:58:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.173+0000 7f86937fe700 1 -- 172.21.15.7:0/3214268543 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f86a4055320 con 0x7f86ac159960 2023-10-07T07:58:34.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8698066f30 msgr2=0x7f86980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8698066f30 0x7f86980693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f869c006fd0 tx=0x7f869c008040 comp rx=0 tx=0).stop 2023-10-07T07:58:34.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac159960 msgr2=0x7f86ac0ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac159960 0x7f86ac0ad8a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f86a400cc90 tx=0x7f86a4001a80 comp rx=0 tx=0).stop 2023-10-07T07:58:34.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 shutdown_connections 2023-10-07T07:58:34.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8698066f30 0x7f86980693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86ac159960 0x7f86ac0ad8a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86ac06d730 0x7f86ac16a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 --2- 172.21.15.7:0/3214268543 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86ac1533a0 0x7f86ac16c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 >> 172.21.15.7:0/3214268543 conn(0x7f86ac067a80 msgr2=0x7f86ac0ca160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:34.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 shutdown_connections 2023-10-07T07:58:34.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.175+0000 7f86b1bd8700 1 -- 172.21.15.7:0/3214268543 wait complete. 2023-10-07T07:58:34.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-07T07:58:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.322+0000 7f009539a700 1 Processor -- start 2023-10-07T07:58:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.322+0000 7f009539a700 1 -- start start 2023-10-07T07:58:34.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.323+0000 7f009539a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00900d0250 0x7f00900d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.323+0000 7f009539a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00900b61f0 con 0x7f00900b3b60 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.323+0000 7f009539a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00900d0be0 con 0x7f0090159380 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.323+0000 7f009539a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00900b4820 con 0x7f00900d0250 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.323+0000 7f008f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f00900b3b60 0x7f00900b3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55154/0 (socket says 172.21.15.7:55154) 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.323+0000 7f008f7fe700 1 -- 172.21.15.7:0/362118683 learned_addr learned my addr 172.21.15.7:0/362118683 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.323+0000 7f008e7fc700 1 --2- 172.21.15.7:0/362118683 >> v2:172.21.15.172:3300/0 conn(0x7f00900d0250 0x7f00900d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.323+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396563207 0 0) 0x7f00900b61f0 con 0x7f00900b3b60 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.323+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f007c003570 con 0x7f00900b3b60 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515038631 0 0) 0x7f00900d0be0 con 0x7f0090159380 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00900b61f0 con 0x7f0090159380 2023-10-07T07:58:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3611005880 0 0) 0x7f007c003570 con 0x7f00900b3b60 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 >> v2:172.21.15.172:3300/0 conn(0x7f00900d0250 msgr2=0x7f00900d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 --2- 172.21.15.7:0/362118683 >> v2:172.21.15.172:3300/0 conn(0x7f00900d0250 0x7f00900d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 >> v1:172.21.15.172:6789/0 conn(0x7f0090159380 legacy=0x7f00900b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900b5000 con 0x7f00900b3b60 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f00880017f0 con 0x7f00900b3b60 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 >> v1:172.21.15.7:6789/0 conn(0x7f00900b3b60 legacy=0x7f00900b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f007c003c40 0x7f007c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f007c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f007c005210 0x7f007c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900b4820 con 0x7f007c004810 2023-10-07T07:58:34.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c00c130 con 0x7f007c003c40 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c00c2c0 con 0x7f007c005210 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008e7fc700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f007c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008effd700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f007c003c40 0x7f007c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008f7fe700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f007c005210 0x7f007c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008e7fc700 1 -- 172.21.15.7:0/362118683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f007c005210 msgr2=0x7f007c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008e7fc700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f007c005210 0x7f007c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008e7fc700 1 -- 172.21.15.7:0/362118683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f007c003c40 msgr2=0x7f007c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008e7fc700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f007c003c40 0x7f007c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008e7fc700 1 -- 172.21.15.7:0/362118683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c003570 con 0x7f007c004810 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008effd700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f007c003c40 0x7f007c004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008f7fe700 1 -- 172.21.15.7:0/362118683 reap_dead start 2023-10-07T07:58:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.324+0000 7f008e7fc700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f007c004c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0078009c30 tx=0x7f00780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19df4e843fbadbfe server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780095b0 con 0x7f007c004810 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007800e710 con 0x7f007c004810 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f008dffb700 1 -- 172.21.15.7:0/362118683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007800ead0 con 0x7f007c004810 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f009539a700 1 -- 172.21.15.7:0/362118683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 msgr2=0x7f007c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f009539a700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f007c004c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0078009c30 tx=0x7f00780091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f009539a700 1 -- 172.21.15.7:0/362118683 shutdown_connections 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f009539a700 1 --2- 172.21.15.7:0/362118683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f007c004c80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f009539a700 1 -- 172.21.15.7:0/362118683 >> 172.21.15.7:0/362118683 conn(0x7f0090067be0 msgr2=0x7f0090067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f009539a700 1 -- 172.21.15.7:0/362118683 shutdown_connections 2023-10-07T07:58:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.325+0000 7f009539a700 1 -- 172.21.15.7:0/362118683 wait complete. 2023-10-07T07:58:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.327+0000 7f009539a700 1 Processor -- start 2023-10-07T07:58:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.327+0000 7f009539a700 1 -- start start 2023-10-07T07:58:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.327+0000 7f009539a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f009016a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.327+0000 7f009539a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f009015a720 0x7f009015ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f009539a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00900c7ae0 0x7f00900c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f009539a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009015c280 con 0x7f007c004810 2023-10-07T07:58:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f009539a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900c8490 con 0x7f009015a720 2023-10-07T07:58:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f009539a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009015b1e0 con 0x7f00900c7ae0 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f009016a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f009016a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34132/0 (socket says 172.21.15.7:34132) 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008effd700 1 -- 172.21.15.7:0/2210986193 learned_addr learned my addr 172.21.15.7:0/2210986193 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008f7fe700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00900c7ae0 0x7f00900c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008e7fc700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f009015a720 0x7f009015ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008effd700 1 -- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00900c7ae0 msgr2=0x7f00900c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008effd700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00900c7ae0 0x7f00900c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008effd700 1 -- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f009015a720 msgr2=0x7f009015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008effd700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f009015a720 0x7f009015ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008effd700 1 -- 172.21.15.7:0/2210986193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009015b4c0 con 0x7f007c004810 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f008effd700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f009016a4d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f008000e870 tx=0x7f0080009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f0086ffd700 1 -- 172.21.15.7:0/2210986193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080006820 con 0x7f007c004810 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f0086ffd700 1 -- 172.21.15.7:0/2210986193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0080006980 con 0x7f007c004810 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f0086ffd700 1 -- 172.21.15.7:0/2210986193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080006d40 con 0x7f007c004810 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009015b7a0 con 0x7f007c004810 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.328+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009015bd60 con 0x7f007c004810 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.329+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00900cf900 con 0x7f007c004810 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.330+0000 7f0086ffd700 1 -- 172.21.15.7:0/2210986193 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0080007240 con 0x7f007c004810 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.330+0000 7f0086ffd700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0070066f30 0x7f00700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.330+0000 7f0086ffd700 1 -- 172.21.15.7:0/2210986193 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0080084590 con 0x7f007c004810 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.331+0000 7f008e7fc700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0070066f30 0x7f00700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.331+0000 7f008e7fc700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0070066f30 0x7f00700693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f009015cc00 tx=0x7f007801a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:34.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.335+0000 7f0086ffd700 1 -- 172.21.15.7:0/2210986193 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0080051b50 con 0x7f007c004810 2023-10-07T07:58:34.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.619+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f009015bf80 con 0x7f007c004810 2023-10-07T07:58:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.622+0000 7f0086ffd700 1 -- 172.21.15.7:0/2210986193 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0080055170 con 0x7f007c004810 2023-10-07T07:58:34.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0070066f30 msgr2=0x7f00700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0070066f30 0x7f00700693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f009015cc00 tx=0x7f007801a000 comp rx=0 tx=0).stop 2023-10-07T07:58:34.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 msgr2=0x7f009016a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f009016a4d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f008000e870 tx=0x7f0080009040 comp rx=0 tx=0).stop 2023-10-07T07:58:34.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 shutdown_connections 2023-10-07T07:58:34.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0070066f30 0x7f00700693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00900c7ae0 0x7f00900c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c004810 0x7f009016a4d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 --2- 172.21.15.7:0/2210986193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f009015a720 0x7f009015ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 >> 172.21.15.7:0/2210986193 conn(0x7f0090067be0 msgr2=0x7f0090157770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:34.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 shutdown_connections 2023-10-07T07:58:34.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.624+0000 7f009539a700 1 -- 172.21.15.7:0/2210986193 wait complete. 2023-10-07T07:58:34.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-07T07:58:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.771+0000 7febe238d700 1 Processor -- start 2023-10-07T07:58:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.771+0000 7febe238d700 1 -- start start 2023-10-07T07:58:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.772+0000 7febe238d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7febdc159310 0x7febdc0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.772+0000 7febe238d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febdc068100 con 0x7febdc1596e0 2023-10-07T07:58:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.772+0000 7febe238d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febdc068350 con 0x7febdc0ced30 2023-10-07T07:58:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.772+0000 7febe238d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febdc068630 con 0x7febdc159310 2023-10-07T07:58:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.772+0000 7febe092a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7febdc1596e0 0x7febdc06b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55158/0 (socket says 172.21.15.7:55158) 2023-10-07T07:58:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.772+0000 7febe092a700 1 -- 172.21.15.7:0/3390479246 learned_addr learned my addr 172.21.15.7:0/3390479246 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdbfff700 1 --2- 172.21.15.7:0/3390479246 >> v2:172.21.15.172:3300/0 conn(0x7febdc159310 0x7febdc0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134429466 0 0) 0x7febdc068100 con 0x7febdc1596e0 2023-10-07T07:58:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc8003570 con 0x7febdc1596e0 2023-10-07T07:58:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 324555358 0 0) 0x7febc8003570 con 0x7febdc1596e0 2023-10-07T07:58:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 >> v2:172.21.15.172:3300/0 conn(0x7febdc159310 msgr2=0x7febdc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 --2- 172.21.15.7:0/3390479246 >> v2:172.21.15.172:3300/0 conn(0x7febdc159310 0x7febdc0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 >> v1:172.21.15.172:6789/0 conn(0x7febdc0ced30 legacy=0x7febdc0c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc068910 con 0x7febdc1596e0 2023-10-07T07:58:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7febd40017f0 con 0x7febdc1596e0 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 >> v1:172.21.15.7:6789/0 conn(0x7febdc1596e0 legacy=0x7febdc06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febc8003a60 0x7febc8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febc8004810 0x7febc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc80051c0 0x7febc800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc068630 con 0x7febc80051c0 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc800c0d0 con 0x7febc8003a60 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc800c290 con 0x7febc8004810 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febe092a700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc80051c0 0x7febc800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdbfff700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febc8003a60 0x7febc8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.773+0000 7febdb7fe700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febc8004810 0x7febc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febdbfff700 1 -- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febc8004810 msgr2=0x7febc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:34.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febdbfff700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febc8004810 0x7febc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febdbfff700 1 -- 172.21.15.7:0/3390479246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc80051c0 msgr2=0x7febc800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:34.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febdbfff700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc80051c0 0x7febc800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febdbfff700 1 -- 172.21.15.7:0/3390479246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc8003570 con 0x7febc8003a60 2023-10-07T07:58:34.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febe092a700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc80051c0 0x7febc800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:34.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febe092a700 1 -- 172.21.15.7:0/3390479246 reap_dead start 2023-10-07T07:58:34.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febdbfff700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febc8003a60 0x7febc8003e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7febcc009c00 tx=0x7febcc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1447640f64d7316f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:34.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00e710 con 0x7febc8003a60 2023-10-07T07:58:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00e870 con 0x7febc8003a60 2023-10-07T07:58:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.774+0000 7febdaffd700 1 -- 172.21.15.7:0/3390479246 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00ec30 con 0x7febc8003a60 2023-10-07T07:58:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.775+0000 7febe238d700 1 -- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febc8003a60 msgr2=0x7febc8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.775+0000 7febe238d700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febc8003a60 0x7febc8003e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7febcc009c00 tx=0x7febcc0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.775+0000 7febe238d700 1 -- 172.21.15.7:0/3390479246 shutdown_connections 2023-10-07T07:58:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.775+0000 7febe238d700 1 --2- 172.21.15.7:0/3390479246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febc8003a60 0x7febc8003e70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.775+0000 7febe238d700 1 -- 172.21.15.7:0/3390479246 >> 172.21.15.7:0/3390479246 conn(0x7febdc067ac0 msgr2=0x7febdc0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.775+0000 7febe238d700 1 -- 172.21.15.7:0/3390479246 shutdown_connections 2023-10-07T07:58:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.775+0000 7febe238d700 1 -- 172.21.15.7:0/3390479246 wait complete. 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.776+0000 7febe238d700 1 Processor -- start 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.777+0000 7febe238d700 1 -- start start 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.777+0000 7febe238d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8003a60 0x7febdc16d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febe238d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febdc0c7e40 0x7febdc0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febe238d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febdc1394f0 0x7febdc139990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febe238d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc139ed0 con 0x7febc8003a60 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febe238d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc16d970 con 0x7febdc0c7e40 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febe238d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0c79b0 con 0x7febdc1394f0 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8003a60 0x7febdc16d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8003a60 0x7febdc16d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34158/0 (socket says 172.21.15.7:34158) 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdbfff700 1 -- 172.21.15.7:0/3774728175 learned_addr learned my addr 172.21.15.7:0/3774728175 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febe092a700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febdc1394f0 0x7febdc139990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdb7fe700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febdc0c7e40 0x7febdc0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdbfff700 1 -- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febdc1394f0 msgr2=0x7febdc139990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdbfff700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febdc1394f0 0x7febdc139990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdbfff700 1 -- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febdc0c7e40 msgr2=0x7febdc0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdbfff700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febdc0c7e40 0x7febdc0b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdbfff700 1 -- 172.21.15.7:0/3774728175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc0c6da0 con 0x7febc8003a60 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febe092a700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febdc1394f0 0x7febdc139990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdb7fe700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febdc0c7e40 0x7febdc0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febdbfff700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8003a60 0x7febdc16d430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7febc800c260 tx=0x7febcc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febd3fff700 1 -- 172.21.15.7:0/3774728175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc022070 con 0x7febc8003a60 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febd3fff700 1 -- 172.21.15.7:0/3774728175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00e9e0 con 0x7febc8003a60 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc0c7080 con 0x7febc8003a60 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febd3fff700 1 -- 172.21.15.7:0/3774728175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc01c450 con 0x7febc8003a60 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.778+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc0c7640 con 0x7febc8003a60 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.780+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc0716c0 con 0x7febc8003a60 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.780+0000 7febd3fff700 1 -- 172.21.15.7:0/3774728175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febcc01c840 con 0x7febc8003a60 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.780+0000 7febd3fff700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7febbc066f30 0x7febbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.781+0000 7febd3fff700 1 -- 172.21.15.7:0/3774728175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7febcc0858c0 con 0x7febc8003a60 2023-10-07T07:58:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.781+0000 7febdb7fe700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7febbc066f30 0x7febbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.781+0000 7febdb7fe700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7febbc066f30 0x7febbc0693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7febc4000f60 tx=0x7febc4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:34.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:34.785+0000 7febd3fff700 1 -- 172.21.15.7:0/3774728175 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febcc052f30 con 0x7febc8003a60 2023-10-07T07:58:35.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.070+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7febdc0cea10 con 0x7febc8003a60 2023-10-07T07:58:35.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.073+0000 7febd3fff700 1 -- 172.21.15.7:0/3774728175 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7febcc006cb0 con 0x7febc8003a60 2023-10-07T07:58:35.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7febbc066f30 msgr2=0x7febbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7febbc066f30 0x7febbc0693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7febc4000f60 tx=0x7febc4009040 comp rx=0 tx=0).stop 2023-10-07T07:58:35.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8003a60 msgr2=0x7febdc16d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8003a60 0x7febdc16d430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7febc800c260 tx=0x7febcc01a000 comp rx=0 tx=0).stop 2023-10-07T07:58:35.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 shutdown_connections 2023-10-07T07:58:35.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7febbc066f30 0x7febbc0693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7febdc1394f0 0x7febdc139990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8003a60 0x7febdc16d430 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 --2- 172.21.15.7:0/3774728175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febdc0c7e40 0x7febdc0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 >> 172.21.15.7:0/3774728175 conn(0x7febdc067ac0 msgr2=0x7febdc13ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 shutdown_connections 2023-10-07T07:58:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.076+0000 7febe238d700 1 -- 172.21.15.7:0/3774728175 wait complete. 2023-10-07T07:58:35.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-07T07:58:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.223+0000 7fb73ea36700 1 Processor -- start 2023-10-07T07:58:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.223+0000 7fb73ea36700 1 -- start start 2023-10-07T07:58:35.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.224+0000 7fb73ea36700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb738159380 0x7fb7380b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.224+0000 7fb73ea36700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7380b4820 con 0x7fb7380b4270 2023-10-07T07:58:35.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.224+0000 7fb73ea36700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7380b5000 con 0x7fb7380d0250 2023-10-07T07:58:35.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.224+0000 7fb73ea36700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7380b52e0 con 0x7fb738159380 2023-10-07T07:58:35.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.224+0000 7fb737fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb738159380 0x7fb7380b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.224+0000 7fb73cfd3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb7380b4270 0x7fb7380ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55172/0 (socket says 172.21.15.7:55172) 2023-10-07T07:58:35.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.224+0000 7fb73cfd3700 1 -- 172.21.15.7:0/461262118 learned_addr learned my addr 172.21.15.7:0/461262118 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:35.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.225+0000 7fb737fff700 1 -- 172.21.15.7:0/461262118 >> v1:172.21.15.172:6789/0 conn(0x7fb7380d0250 legacy=0x7fb7380b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:35.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.225+0000 7fb737fff700 1 -- 172.21.15.7:0/461262118 >> v1:172.21.15.7:6789/0 conn(0x7fb7380b4270 legacy=0x7fb7380ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.225+0000 7fb737fff700 1 -- 172.21.15.7:0/461262118 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380b55c0 con 0x7fb738159380 2023-10-07T07:58:35.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.225+0000 7fb737fff700 1 --2- 172.21.15.7:0/461262118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 0x7fb7380b5ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb7200098e0 tx=0x7fb7200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a7418c02e5ec3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:35.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.225+0000 7fb736ffd700 1 -- 172.21.15.7:0/461262118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720009e80 con 0x7fb738159380 2023-10-07T07:58:35.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.225+0000 7fb736ffd700 1 -- 172.21.15.7:0/461262118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72000b820 con 0x7fb738159380 2023-10-07T07:58:35.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.225+0000 7fb736ffd700 1 -- 172.21.15.7:0/461262118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72000bba0 con 0x7fb738159380 2023-10-07T07:58:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.225+0000 7fb73ea36700 1 -- 172.21.15.7:0/461262118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 msgr2=0x7fb7380b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.225+0000 7fb73ea36700 1 --2- 172.21.15.7:0/461262118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 0x7fb7380b5ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb7200098e0 tx=0x7fb7200091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.226+0000 7fb73ea36700 1 -- 172.21.15.7:0/461262118 shutdown_connections 2023-10-07T07:58:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.226+0000 7fb73ea36700 1 --2- 172.21.15.7:0/461262118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 0x7fb7380b5ae0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.226+0000 7fb73ea36700 1 -- 172.21.15.7:0/461262118 >> 172.21.15.7:0/461262118 conn(0x7fb738067be0 msgr2=0x7fb738067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.226+0000 7fb73ea36700 1 -- 172.21.15.7:0/461262118 shutdown_connections 2023-10-07T07:58:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.226+0000 7fb73ea36700 1 -- 172.21.15.7:0/461262118 wait complete. 2023-10-07T07:58:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.227+0000 7fb73ea36700 1 Processor -- start 2023-10-07T07:58:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.227+0000 7fb73ea36700 1 -- start start 2023-10-07T07:58:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73ea36700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7380b4270 0x7fb738153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73ea36700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7380d0250 0x7fb738150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73ea36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 0x7fb738156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73ea36700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7381542f0 con 0x7fb738159380 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73ea36700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb738151a30 con 0x7fb7380d0250 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73ea36700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb738151d10 con 0x7fb7380b4270 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73cfd3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 0x7fb738156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73cfd3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 0x7fb738156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34178/0 (socket says 172.21.15.7:34178) 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73cfd3700 1 -- 172.21.15.7:0/4453109 learned_addr learned my addr 172.21.15.7:0/4453109 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb7377fe700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7380d0250 0x7fb738150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb737fff700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7380b4270 0x7fb738153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73cfd3700 1 -- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7380b4270 msgr2=0x7fb738153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73cfd3700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7380b4270 0x7fb738153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73cfd3700 1 -- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7380d0250 msgr2=0x7fb738150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73cfd3700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7380d0250 0x7fb738150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb73cfd3700 1 -- 172.21.15.7:0/4453109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb720009420 con 0x7fb738159380 2023-10-07T07:58:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.228+0000 7fb737fff700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7380b4270 0x7fb738153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.229+0000 7fb73cfd3700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 0x7fb738156c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb73000cc40 tx=0x7fb730001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.229+0000 7fb72ffff700 1 -- 172.21.15.7:0/4453109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000a910 con 0x7fb738159380 2023-10-07T07:58:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.229+0000 7fb72ffff700 1 -- 172.21.15.7:0/4453109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73000aa70 con 0x7fb738159380 2023-10-07T07:58:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.229+0000 7fb72ffff700 1 -- 172.21.15.7:0/4453109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000ae30 con 0x7fb738159380 2023-10-07T07:58:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.229+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7381532d0 con 0x7fb738159380 2023-10-07T07:58:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.229+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb738153890 con 0x7fb738159380 2023-10-07T07:58:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.230+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb738153ab0 con 0x7fb738159380 2023-10-07T07:58:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.231+0000 7fb72ffff700 1 -- 172.21.15.7:0/4453109 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb730007300 con 0x7fb738159380 2023-10-07T07:58:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.231+0000 7fb72ffff700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb718066f30 0x7fb7180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.231+0000 7fb72ffff700 1 -- 172.21.15.7:0/4453109 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb730084500 con 0x7fb738159380 2023-10-07T07:58:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.231+0000 7fb737fff700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb718066f30 0x7fb7180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.232+0000 7fb737fff700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb718066f30 0x7fb7180693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7200030c0 tx=0x7fb720016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:35.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.236+0000 7fb72ffff700 1 -- 172.21.15.7:0/4453109 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb730051b40 con 0x7fb738159380 2023-10-07T07:58:35.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.515+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fb7380716c0 con 0x7fb738159380 2023-10-07T07:58:35.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.518+0000 7fb72ffff700 1 -- 172.21.15.7:0/4453109 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb730055160 con 0x7fb738159380 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb718066f30 msgr2=0x7fb7180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb718066f30 0x7fb7180693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7200030c0 tx=0x7fb720016000 comp rx=0 tx=0).stop 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 msgr2=0x7fb738156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 0x7fb738156c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb73000cc40 tx=0x7fb730001a50 comp rx=0 tx=0).stop 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 shutdown_connections 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb718066f30 0x7fb7180693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb738159380 0x7fb738156c20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7380b4270 0x7fb738153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 --2- 172.21.15.7:0/4453109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7380d0250 0x7fb738150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.520+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 >> 172.21.15.7:0/4453109 conn(0x7fb738067be0 msgr2=0x7fb7381528b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.521+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 shutdown_connections 2023-10-07T07:58:35.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.521+0000 7fb73ea36700 1 -- 172.21.15.7:0/4453109 wait complete. 2023-10-07T07:58:35.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-07T07:58:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.667+0000 7f55df675700 1 Processor -- start 2023-10-07T07:58:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.667+0000 7f55df675700 1 -- start start 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55df675700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f55d8156940 0x7f55d806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55df675700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f55d8063230 0x7f55d814fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55df675700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 0x7f55d80c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55df675700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d80af590 con 0x7f55d8063230 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55df675700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d80b0b30 con 0x7f55d8156940 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55df675700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d806e1f0 con 0x7f55d80636a0 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55dcc10700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f55d8063230 0x7f55d814fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55dcc10700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f55d8063230 0x7f55d814fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34180/0 (socket says 172.21.15.7:34180) 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55dcc10700 1 -- 172.21.15.7:0/3916778321 learned_addr learned my addr 172.21.15.7:0/3916778321 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55dd411700 1 --2- 172.21.15.7:0/3916778321 >> v2:172.21.15.172:3300/0 conn(0x7f55d8156940 0x7f55d806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.668+0000 7f55ddc12700 1 --2- 172.21.15.7:0/3916778321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 0x7f55d80c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55dcc10700 1 -- 172.21.15.7:0/3916778321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 msgr2=0x7f55d80c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55dcc10700 1 --2- 172.21.15.7:0/3916778321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 0x7f55d80c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55dcc10700 1 -- 172.21.15.7:0/3916778321 >> v2:172.21.15.172:3300/0 conn(0x7f55d8156940 msgr2=0x7f55d806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55dcc10700 1 --2- 172.21.15.7:0/3916778321 >> v2:172.21.15.172:3300/0 conn(0x7f55d8156940 0x7f55d806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55dcc10700 1 -- 172.21.15.7:0/3916778321 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d806e4d0 con 0x7f55d8063230 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55dd411700 1 --2- 172.21.15.7:0/3916778321 >> v2:172.21.15.172:3300/0 conn(0x7f55d8156940 0x7f55d806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55dcc10700 1 --2- 172.21.15.7:0/3916778321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 0x7f55d814fb60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f55d806d240 tx=0x7f55c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f356c717befaa0ae server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55cffff700 1 -- 172.21.15.7:0/3916778321 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c800b860 con 0x7f55d8063230 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55cffff700 1 -- 172.21.15.7:0/3916778321 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c800b9c0 con 0x7f55d8063230 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55cffff700 1 -- 172.21.15.7:0/3916778321 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c800bd60 con 0x7f55d8063230 2023-10-07T07:58:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55df675700 1 -- 172.21.15.7:0/3916778321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 msgr2=0x7f55d814fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.669+0000 7f55df675700 1 --2- 172.21.15.7:0/3916778321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 0x7f55d814fb60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f55d806d240 tx=0x7f55c8009250 comp rx=0 tx=0).stop 2023-10-07T07:58:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.670+0000 7f55df675700 1 -- 172.21.15.7:0/3916778321 shutdown_connections 2023-10-07T07:58:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.670+0000 7f55df675700 1 --2- 172.21.15.7:0/3916778321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 0x7f55d80c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.670+0000 7f55df675700 1 --2- 172.21.15.7:0/3916778321 >> v2:172.21.15.172:3300/0 conn(0x7f55d8156940 0x7f55d806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.670+0000 7f55df675700 1 --2- 172.21.15.7:0/3916778321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 0x7f55d814fb60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.670+0000 7f55df675700 1 -- 172.21.15.7:0/3916778321 >> 172.21.15.7:0/3916778321 conn(0x7f55d80cd1a0 msgr2=0x7f55d80c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.670+0000 7f55df675700 1 -- 172.21.15.7:0/3916778321 shutdown_connections 2023-10-07T07:58:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.670+0000 7f55df675700 1 -- 172.21.15.7:0/3916778321 wait complete. 2023-10-07T07:58:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.671+0000 7f55df675700 1 Processor -- start 2023-10-07T07:58:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.671+0000 7f55df675700 1 -- start start 2023-10-07T07:58:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55df675700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 0x7f55d80c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55df675700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 0x7f55d814e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55df675700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d8156940 0x7f55d816a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55df675700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d80c2620 con 0x7f55d8063230 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55df675700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d80c2ab0 con 0x7f55d8156940 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55df675700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d80c1a10 con 0x7f55d80636a0 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dd411700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 0x7f55d80c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dd411700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 0x7f55d80c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34192/0 (socket says 172.21.15.7:34192) 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dd411700 1 -- 172.21.15.7:0/4115628851 learned_addr learned my addr 172.21.15.7:0/4115628851 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dcc10700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 0x7f55d814e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dd411700 1 -- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 msgr2=0x7f55d814e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55ddc12700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d8156940 0x7f55d816a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dd411700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 0x7f55d814e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dd411700 1 -- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d8156940 msgr2=0x7f55d816a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dd411700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d8156940 0x7f55d816a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dd411700 1 -- 172.21.15.7:0/4115628851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c80094f0 con 0x7f55d8063230 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55dcc10700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 0x7f55d814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.672+0000 7f55ddc12700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d8156940 0x7f55d816a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.673+0000 7f55dd411700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 0x7f55d80c4060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f55d400e8d0 tx=0x7f55d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.673+0000 7f55cd7fa700 1 -- 172.21.15.7:0/4115628851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d400ee70 con 0x7f55d8063230 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.673+0000 7f55cd7fa700 1 -- 172.21.15.7:0/4115628851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d400c5e0 con 0x7f55d8063230 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.673+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d80c1cf0 con 0x7f55d8063230 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.673+0000 7f55cd7fa700 1 -- 172.21.15.7:0/4115628851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d400c980 con 0x7f55d8063230 2023-10-07T07:58:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.673+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d80c22b0 con 0x7f55d8063230 2023-10-07T07:58:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.674+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d80716c0 con 0x7f55d8063230 2023-10-07T07:58:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.675+0000 7f55cd7fa700 1 -- 172.21.15.7:0/4115628851 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55d4025480 con 0x7f55d8063230 2023-10-07T07:58:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.675+0000 7f55cd7fa700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55bc066f30 0x7f55bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.675+0000 7f55cd7fa700 1 -- 172.21.15.7:0/4115628851 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f55d40844e0 con 0x7f55d8063230 2023-10-07T07:58:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.675+0000 7f55dcc10700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55bc066f30 0x7f55bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.676+0000 7f55dcc10700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55bc066f30 0x7f55bc0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55d806d240 tx=0x7f55c8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.679+0000 7f55cd7fa700 1 -- 172.21.15.7:0/4115628851 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55d4055510 con 0x7f55d8063230 2023-10-07T07:58:35.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.971+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f55d80c2900 con 0x7f55d8063230 2023-10-07T07:58:35.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.975+0000 7f55cd7fa700 1 -- 172.21.15.7:0/4115628851 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f55d40200a0 con 0x7f55d8063230 2023-10-07T07:58:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.977+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55bc066f30 msgr2=0x7f55bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.977+0000 7f55df675700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55bc066f30 0x7f55bc0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55d806d240 tx=0x7f55c8017000 comp rx=0 tx=0).stop 2023-10-07T07:58:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.977+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 msgr2=0x7f55d80c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.977+0000 7f55df675700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 0x7f55d80c4060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f55d400e8d0 tx=0x7f55d4008040 comp rx=0 tx=0).stop 2023-10-07T07:58:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.978+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 shutdown_connections 2023-10-07T07:58:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.978+0000 7f55df675700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55bc066f30 0x7f55bc0693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.978+0000 7f55df675700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d8156940 0x7f55d816a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.978+0000 7f55df675700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55d8063230 0x7f55d80c4060 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.978+0000 7f55df675700 1 --2- 172.21.15.7:0/4115628851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55d80636a0 0x7f55d814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.978+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 >> 172.21.15.7:0/4115628851 conn(0x7f55d80cd1a0 msgr2=0x7f55d8159130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.978+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 shutdown_connections 2023-10-07T07:58:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:35.978+0000 7f55df675700 1 -- 172.21.15.7:0/4115628851 wait complete. 2023-10-07T07:58:35.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-07T07:58:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.124+0000 7fcc34332700 1 Processor -- start 2023-10-07T07:58:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.124+0000 7fcc34332700 1 -- start start 2023-10-07T07:58:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.125+0000 7fcc34332700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc2c0ced30 0x7fcc2c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.125+0000 7fcc34332700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c068100 con 0x7fcc2c1596e0 2023-10-07T07:58:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.125+0000 7fcc34332700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c068350 con 0x7fcc2c159310 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.125+0000 7fcc34332700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c068630 con 0x7fcc2c0ced30 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.125+0000 7fcc328cf700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcc2c1596e0 0x7fcc2c06b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55174/0 (socket says 172.21.15.7:55174) 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc328cf700 1 -- 172.21.15.7:0/1994372017 learned_addr learned my addr 172.21.15.7:0/1994372017 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc318cd700 1 --2- 172.21.15.7:0/1994372017 >> v2:172.21.15.172:3300/0 conn(0x7fcc2c0ced30 0x7fcc2c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238594917 0 0) 0x7fcc2c068100 con 0x7fcc2c1596e0 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc18003570 con 0x7fcc2c1596e0 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497608725 0 0) 0x7fcc2c068350 con 0x7fcc2c159310 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc2c068100 con 0x7fcc2c159310 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3937855933 0 0) 0x7fcc18003570 con 0x7fcc2c1596e0 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 >> v2:172.21.15.172:3300/0 conn(0x7fcc2c0ced30 msgr2=0x7fcc2c0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 --2- 172.21.15.7:0/1994372017 >> v2:172.21.15.172:3300/0 conn(0x7fcc2c0ced30 0x7fcc2c0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 >> v1:172.21.15.172:6789/0 conn(0x7fcc2c159310 legacy=0x7fcc2c0c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c068910 con 0x7fcc2c1596e0 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7fcc240017f0 con 0x7fcc2c1596e0 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 >> v1:172.21.15.7:6789/0 conn(0x7fcc2c1596e0 legacy=0x7fcc2c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc18003bf0 0x7fcc18003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc18004810 0x7fcc18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc1800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c068630 con 0x7fcc180051c0 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1800c0b0 con 0x7fcc18004810 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1800c270 con 0x7fcc18003bf0 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc328cf700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc1800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc318cd700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc18004810 0x7fcc18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.126+0000 7fcc320ce700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc18003bf0 0x7fcc18003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc328cf700 1 -- 172.21.15.7:0/1994372017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc18003bf0 msgr2=0x7fcc18003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc328cf700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc18003bf0 0x7fcc18003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc328cf700 1 -- 172.21.15.7:0/1994372017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc18004810 msgr2=0x7fcc18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc328cf700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc18004810 0x7fcc18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc328cf700 1 -- 172.21.15.7:0/1994372017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc18003570 con 0x7fcc180051c0 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc318cd700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc18004810 0x7fcc18004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc328cf700 1 -- 172.21.15.7:0/1994372017 reap_dead start 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc328cf700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc1800ba60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400a8d0 tx=0x7fcc24002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7a23e2a7abe56e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc240070d0 con 0x7fcc180051c0 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24007270 con 0x7fcc180051c0 2023-10-07T07:58:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.127+0000 7fcc310cc700 1 -- 172.21.15.7:0/1994372017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24007630 con 0x7fcc180051c0 2023-10-07T07:58:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.128+0000 7fcc34332700 1 -- 172.21.15.7:0/1994372017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 msgr2=0x7fcc1800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.128+0000 7fcc34332700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc1800ba60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400a8d0 tx=0x7fcc24002190 comp rx=0 tx=0).stop 2023-10-07T07:58:36.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.128+0000 7fcc34332700 1 -- 172.21.15.7:0/1994372017 shutdown_connections 2023-10-07T07:58:36.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.128+0000 7fcc34332700 1 --2- 172.21.15.7:0/1994372017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc1800ba60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.128+0000 7fcc34332700 1 -- 172.21.15.7:0/1994372017 >> 172.21.15.7:0/1994372017 conn(0x7fcc2c067ac0 msgr2=0x7fcc2c0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.128+0000 7fcc34332700 1 -- 172.21.15.7:0/1994372017 shutdown_connections 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.128+0000 7fcc34332700 1 -- 172.21.15.7:0/1994372017 wait complete. 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.129+0000 7fcc34332700 1 Processor -- start 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.129+0000 7fcc34332700 1 -- start start 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc34332700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc2c13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc34332700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2c15e8d0 0x7fcc2c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc34332700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc2c0b2e50 0x7fcc2c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc34332700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c15d050 con 0x7fcc180051c0 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc34332700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c15dbd0 con 0x7fcc2c0b2e50 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc34332700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0b37e0 con 0x7fcc2c15e8d0 2023-10-07T07:58:36.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc320ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc2c13aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc320ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc2c13aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34208/0 (socket says 172.21.15.7:34208) 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc320ce700 1 -- 172.21.15.7:0/18868185 learned_addr learned my addr 172.21.15.7:0/18868185 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc318cd700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2c15e8d0 0x7fcc2c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc328cf700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc2c0b2e50 0x7fcc2c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc320ce700 1 -- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2c15e8d0 msgr2=0x7fcc2c157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc320ce700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2c15e8d0 0x7fcc2c157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc320ce700 1 -- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc2c0b2e50 msgr2=0x7fcc2c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc320ce700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc2c0b2e50 0x7fcc2c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc320ce700 1 -- 172.21.15.7:0/18868185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c13af40 con 0x7fcc180051c0 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.130+0000 7fcc318cd700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2c15e8d0 0x7fcc2c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.131+0000 7fcc320ce700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc2c13aa00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc20002be0 tx=0x7fcc20007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:36.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.131+0000 7fcc1e7fc700 1 -- 172.21.15.7:0/18868185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000f9a0 con 0x7fcc180051c0 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.131+0000 7fcc1e7fc700 1 -- 172.21.15.7:0/18868185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2000fb00 con 0x7fcc180051c0 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.131+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c158340 con 0x7fcc180051c0 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.131+0000 7fcc1e7fc700 1 -- 172.21.15.7:0/18868185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000fec0 con 0x7fcc180051c0 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.131+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c15d580 con 0x7fcc180051c0 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.132+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c0716c0 con 0x7fcc180051c0 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.132+0000 7fcc1e7fc700 1 -- 172.21.15.7:0/18868185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc200133f0 con 0x7fcc180051c0 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.133+0000 7fcc1e7fc700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc10066f30 0x7fcc100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.133+0000 7fcc1e7fc700 1 -- 172.21.15.7:0/18868185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcc20084380 con 0x7fcc180051c0 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.133+0000 7fcc318cd700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc10066f30 0x7fcc100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.133+0000 7fcc318cd700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc10066f30 0x7fcc100693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009720 tx=0x7fcc280091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:36.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.137+0000 7fcc1e7fc700 1 -- 172.21.15.7:0/18868185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc200518a0 con 0x7fcc180051c0 2023-10-07T07:58:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.427+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fcc2c0cea10 con 0x7fcc180051c0 2023-10-07T07:58:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.431+0000 7fcc1e7fc700 1 -- 172.21.15.7:0/18868185 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcc20054ec0 con 0x7fcc180051c0 2023-10-07T07:58:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc10066f30 msgr2=0x7fcc100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc10066f30 0x7fcc100693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009720 tx=0x7fcc280091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 msgr2=0x7fcc2c13aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc2c13aa00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc20002be0 tx=0x7fcc20007d30 comp rx=0 tx=0).stop 2023-10-07T07:58:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 shutdown_connections 2023-10-07T07:58:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc10066f30 0x7fcc100693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc2c0b2e50 0x7fcc2c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc180051c0 0x7fcc2c13aa00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 --2- 172.21.15.7:0/18868185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2c15e8d0 0x7fcc2c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 >> 172.21.15.7:0/18868185 conn(0x7fcc2c067ac0 msgr2=0x7fcc2c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 shutdown_connections 2023-10-07T07:58:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.433+0000 7fcc34332700 1 -- 172.21.15.7:0/18868185 wait complete. 2023-10-07T07:58:36.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.579+0000 7f7ba2628700 1 Processor -- start 2023-10-07T07:58:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.580+0000 7f7ba2628700 1 -- start start 2023-10-07T07:58:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.580+0000 7f7ba2628700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c140a80 0x7f7b9c06ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.580+0000 7f7ba2628700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c06e2c0 con 0x7f7b9c067700 2023-10-07T07:58:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.580+0000 7f7ba2628700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c0c7940 con 0x7f7b9c1582c0 2023-10-07T07:58:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.580+0000 7f7ba2628700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c06f250 con 0x7f7b9c140a80 2023-10-07T07:58:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7ba0bc5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7b9c067700 0x7f7b9c06a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55188/0 (socket says 172.21.15.7:55188) 2023-10-07T07:58:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7ba0bc5700 1 -- 172.21.15.7:0/1317405886 learned_addr learned my addr 172.21.15.7:0/1317405886 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9bfff700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c140a80 0x7f7b9c06ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222850616 0 0) 0x7f7b9c06e2c0 con 0x7f7b9c067700 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b84003570 con 0x7f7b9c067700 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647614850 0 0) 0x7f7b9c0c7940 con 0x7f7b9c1582c0 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b9c06e2c0 con 0x7f7b9c1582c0 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488755106 0 0) 0x7f7b84003570 con 0x7f7b9c067700 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c140a80 msgr2=0x7f7b9c06ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c140a80 0x7f7b9c06ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 >> v1:172.21.15.172:6789/0 conn(0x7f7b9c1582c0 legacy=0x7f7b9c066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c0c7c20 con 0x7f7b9c067700 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f7b940017f0 con 0x7f7b9c067700 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 >> v1:172.21.15.7:6789/0 conn(0x7f7b9c067700 legacy=0x7f7b9c06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c140a80 0x7f7b84003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b84004180 0x7f7b84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84004d50 0x7f7b840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c06f250 con 0x7f7b84004d50 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8400bba0 con 0x7f7b84004180 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.581+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8400bd60 con 0x7f7b9c140a80 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9bfff700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c140a80 0x7f7b84003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7ba0bc5700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84004d50 0x7f7b840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9b7fe700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b84004180 0x7f7b84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9bfff700 1 -- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b84004180 msgr2=0x7f7b84004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9bfff700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b84004180 0x7f7b84004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9bfff700 1 -- 172.21.15.7:0/1317405886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84004d50 msgr2=0x7f7b840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9bfff700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b84004d50 0x7f7b840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9bfff700 1 -- 172.21.15.7:0/1317405886 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84003570 con 0x7f7b9c140a80 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9bfff700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c140a80 0x7f7b84003c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009c00 tx=0x7f7b8c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7affc3ba6f466f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00e710 con 0x7f7b9c140a80 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00e870 con 0x7f7b9c140a80 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.582+0000 7f7b9affd700 1 -- 172.21.15.7:0/1317405886 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00ec30 con 0x7f7b9c140a80 2023-10-07T07:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.583+0000 7f7ba2628700 1 -- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c140a80 msgr2=0x7f7b84003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.583+0000 7f7ba2628700 1 --2- 172.21.15.7:0/1317405886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c140a80 0x7f7b84003c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009c00 tx=0x7f7b8c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.583+0000 7f7ba0bc5700 1 -- 172.21.15.7:0/1317405886 reap_dead start 2023-10-07T07:58:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.583+0000 7f7ba2628700 1 -- 172.21.15.7:0/1317405886 shutdown_connections 2023-10-07T07:58:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.583+0000 7f7ba2628700 1 -- 172.21.15.7:0/1317405886 >> 172.21.15.7:0/1317405886 conn(0x7f7b9c068bd0 msgr2=0x7f7b9c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.583+0000 7f7ba2628700 1 -- 172.21.15.7:0/1317405886 shutdown_connections 2023-10-07T07:58:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.583+0000 7f7ba2628700 1 -- 172.21.15.7:0/1317405886 wait complete. 2023-10-07T07:58:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.585+0000 7f7ba2628700 1 Processor -- start 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.585+0000 7f7ba2628700 1 -- start start 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7ba2628700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c156fb0 0x7f7b9c0c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7ba2628700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c157380 0x7f7b9c0c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7ba2628700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9c12c320 0x7f7b9c12c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7ba2628700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0c73b0 con 0x7f7b9c157380 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7ba2628700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0c6f20 con 0x7f7b9c12c320 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7ba2628700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0c6310 con 0x7f7b9c156fb0 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c157380 0x7f7b9c0c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c157380 0x7f7b9c0c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34222/0 (socket says 172.21.15.7:34222) 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9b7fe700 1 -- 172.21.15.7:0/851713485 learned_addr learned my addr 172.21.15.7:0/851713485 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9bfff700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c156fb0 0x7f7b9c0c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7ba0bc5700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9c12c320 0x7f7b9c12c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9b7fe700 1 -- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c156fb0 msgr2=0x7f7b9c0c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9b7fe700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c156fb0 0x7f7b9c0c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9b7fe700 1 -- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9c12c320 msgr2=0x7f7b9c12c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9b7fe700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9c12c320 0x7f7b9c12c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9b7fe700 1 -- 172.21.15.7:0/851713485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c0c65f0 con 0x7f7b9c157380 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.586+0000 7f7b9b7fe700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c157380 0x7f7b9c0c5a90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b90002c00 tx=0x7f7b90007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.587+0000 7f7b83fff700 1 -- 172.21.15.7:0/851713485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000fa00 con 0x7f7b9c157380 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.587+0000 7f7b83fff700 1 -- 172.21.15.7:0/851713485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9000fb80 con 0x7f7b9c157380 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.587+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c0c68d0 con 0x7f7b9c157380 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.587+0000 7f7b83fff700 1 -- 172.21.15.7:0/851713485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900133f0 con 0x7f7b9c157380 2023-10-07T07:58:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.587+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c0c5fd0 con 0x7f7b9c157380 2023-10-07T07:58:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.588+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c0716c0 con 0x7f7b9c157380 2023-10-07T07:58:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.588+0000 7f7b83fff700 1 -- 172.21.15.7:0/851713485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b90013760 con 0x7f7b9c157380 2023-10-07T07:58:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.589+0000 7f7b83fff700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7b88066f30 0x7f7b880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.589+0000 7f7b83fff700 1 -- 172.21.15.7:0/851713485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7b900848e0 con 0x7f7b9c157380 2023-10-07T07:58:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.589+0000 7f7b9bfff700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7b88066f30 0x7f7b880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.589+0000 7f7b9bfff700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7b88066f30 0x7f7b880693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0091e0 tx=0x7f7b8c01a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:36.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.593+0000 7f7b83fff700 1 -- 172.21.15.7:0/851713485 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b90051f60 con 0x7f7b9c157380 2023-10-07T07:58:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.798+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b9c0c7060 con 0x7f7b9c157380 2023-10-07T07:58:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.799+0000 7f7b83fff700 1 -- 172.21.15.7:0/851713485 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f7b90055580 con 0x7f7b9c157380 2023-10-07T07:58:36.800 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:36.801 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:58:33.252335+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7b88066f30 msgr2=0x7f7b880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7b88066f30 0x7f7b880693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0091e0 tx=0x7f7b8c01a000 comp rx=0 tx=0).stop 2023-10-07T07:58:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c157380 msgr2=0x7f7b9c0c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c157380 0x7f7b9c0c5a90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b90002c00 tx=0x7f7b90007d30 comp rx=0 tx=0).stop 2023-10-07T07:58:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 shutdown_connections 2023-10-07T07:58:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7b88066f30 0x7f7b880693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9c12c320 0x7f7b9c12c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b9c156fb0 0x7f7b9c0c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 --2- 172.21.15.7:0/851713485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c157380 0x7f7b9c0c5a90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 >> 172.21.15.7:0/851713485 conn(0x7f7b9c068bd0 msgr2=0x7f7b9c0c3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 shutdown_connections 2023-10-07T07:58:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.802+0000 7f7ba2628700 1 -- 172.21.15.7:0/851713485 wait complete. 2023-10-07T07:58:36.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T07:58:36.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:58:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.950+0000 7f76c36d2700 1 Processor -- start 2023-10-07T07:58:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.950+0000 7f76c36d2700 1 -- start start 2023-10-07T07:58:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.951+0000 7f76c36d2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc162170 0x7f76bc15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.951+0000 7f76c36d2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76bc0ca300 con 0x7f76bc068620 2023-10-07T07:58:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.951+0000 7f76c36d2700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76bc0cae80 con 0x7f76bc068250 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.951+0000 7f76c36d2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc06f030 con 0x7f76bc162170 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.951+0000 7f76c1c6f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f76bc068620 0x7f76bc15daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55202/0 (socket says 172.21.15.7:55202) 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.951+0000 7f76c1c6f700 1 -- 172.21.15.7:0/1197458082 learned_addr learned my addr 172.21.15.7:0/1197458082 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.951+0000 7f76c0c6d700 1 --2- 172.21.15.7:0/1197458082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc162170 0x7f76bc15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76b3fff700 1 -- 172.21.15.7:0/1197458082 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256415583 0 0) 0x7f76bc0ca300 con 0x7f76bc068620 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76b3fff700 1 -- 172.21.15.7:0/1197458082 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76a8003570 con 0x7f76bc068620 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76b3fff700 1 -- 172.21.15.7:0/1197458082 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454892610 0 0) 0x7f76bc0cae80 con 0x7f76bc068250 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76b3fff700 1 -- 172.21.15.7:0/1197458082 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76bc0ca300 con 0x7f76bc068250 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76c0c6d700 1 -- 172.21.15.7:0/1197458082 >> v1:172.21.15.172:6789/0 conn(0x7f76bc068250 legacy=0x7f76bc06e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76c0c6d700 1 -- 172.21.15.7:0/1197458082 >> v1:172.21.15.7:6789/0 conn(0x7f76bc068620 legacy=0x7f76bc15daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76c0c6d700 1 -- 172.21.15.7:0/1197458082 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0ca550 con 0x7f76bc162170 2023-10-07T07:58:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76c0c6d700 1 --2- 172.21.15.7:0/1197458082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc162170 0x7f76bc15d560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009910 tx=0x7f76b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6781e848e56c3285 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76b3fff700 1 -- 172.21.15.7:0/1197458082 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8011630 con 0x7f76bc162170 2023-10-07T07:58:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76b3fff700 1 -- 172.21.15.7:0/1197458082 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b8011790 con 0x7f76bc162170 2023-10-07T07:58:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.952+0000 7f76b3fff700 1 -- 172.21.15.7:0/1197458082 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8011b30 con 0x7f76bc162170 2023-10-07T07:58:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.953+0000 7f76c36d2700 1 -- 172.21.15.7:0/1197458082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc162170 msgr2=0x7f76bc15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.953+0000 7f76c36d2700 1 --2- 172.21.15.7:0/1197458082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc162170 0x7f76bc15d560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009910 tx=0x7f76b80091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.953+0000 7f76c36d2700 1 -- 172.21.15.7:0/1197458082 shutdown_connections 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.953+0000 7f76c36d2700 1 --2- 172.21.15.7:0/1197458082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc162170 0x7f76bc15d560 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.953+0000 7f76c36d2700 1 -- 172.21.15.7:0/1197458082 >> 172.21.15.7:0/1197458082 conn(0x7f76bc0c9c30 msgr2=0x7f76bc06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.953+0000 7f76c36d2700 1 -- 172.21.15.7:0/1197458082 shutdown_connections 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.953+0000 7f76c36d2700 1 -- 172.21.15.7:0/1197458082 wait complete. 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.954+0000 7f76c36d2700 1 Processor -- start 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.954+0000 7f76c36d2700 1 -- start start 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.955+0000 7f76c36d2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc068250 0x7f76bc14ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.955+0000 7f76c36d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc068620 0x7f76bc161930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.955+0000 7f76c36d2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76bc162170 0x7f76bc155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.955+0000 7f76c36d2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc1614a0 con 0x7f76bc068620 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.955+0000 7f76c36d2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc161160 con 0x7f76bc162170 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.955+0000 7f76c36d2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc0b1c30 con 0x7f76bc068250 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.955+0000 7f76c0c6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc068620 0x7f76bc161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.955+0000 7f76c0c6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc068620 0x7f76bc161930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34232/0 (socket says 172.21.15.7:34232) 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.955+0000 7f76c0c6d700 1 -- 172.21.15.7:0/137886959 learned_addr learned my addr 172.21.15.7:0/137886959 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c1c6f700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76bc162170 0x7f76bc155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c146e700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc068250 0x7f76bc14ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c0c6d700 1 -- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc068250 msgr2=0x7f76bc14ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c0c6d700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc068250 0x7f76bc14ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c0c6d700 1 -- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76bc162170 msgr2=0x7f76bc155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c0c6d700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76bc162170 0x7f76bc155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c0c6d700 1 -- 172.21.15.7:0/137886959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b8009420 con 0x7f76bc068620 2023-10-07T07:58:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c146e700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc068250 0x7f76bc14ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c0c6d700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc068620 0x7f76bc161930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f76bc06d500 tx=0x7f76b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76b17fa700 1 -- 172.21.15.7:0/137886959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b801f070 con 0x7f76bc068620 2023-10-07T07:58:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76b17fa700 1 -- 172.21.15.7:0/137886959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b8000bc0 con 0x7f76bc068620 2023-10-07T07:58:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76bc0b1f10 con 0x7f76bc068620 2023-10-07T07:58:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76b17fa700 1 -- 172.21.15.7:0/137886959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8011b30 con 0x7f76bc068620 2023-10-07T07:58:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.956+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76bc161e70 con 0x7f76bc068620 2023-10-07T07:58:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.957+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76bc161650 con 0x7f76bc068620 2023-10-07T07:58:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.958+0000 7f76b17fa700 1 -- 172.21.15.7:0/137886959 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f76b8023070 con 0x7f76bc068620 2023-10-07T07:58:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.958+0000 7f76b17fa700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f76a0066f30 0x7f76a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.958+0000 7f76b17fa700 1 -- 172.21.15.7:0/137886959 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f76b8085580 con 0x7f76bc068620 2023-10-07T07:58:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.958+0000 7f76c146e700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f76a0066f30 0x7f76a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:36.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.959+0000 7f76c146e700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f76a0066f30 0x7f76a00693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0c6450 tx=0x7f76ac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:36.963+0000 7f76b17fa700 1 -- 172.21.15.7:0/137886959 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76bc161650 con 0x7f76bc068620 2023-10-07T07:58:37.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.166+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76bc04c3c0 con 0x7f76bc068620 2023-10-07T07:58:37.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.167+0000 7f76b17fa700 1 -- 172.21.15.7:0/137886959 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76b8015020 con 0x7f76bc068620 2023-10-07T07:58:37.168 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:58:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f76a0066f30 msgr2=0x7f76a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f76a0066f30 0x7f76a00693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0c6450 tx=0x7f76ac009040 comp rx=0 tx=0).stop 2023-10-07T07:58:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc068620 msgr2=0x7f76bc161930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc068620 0x7f76bc161930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f76bc06d500 tx=0x7f76b8017000 comp rx=0 tx=0).stop 2023-10-07T07:58:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 shutdown_connections 2023-10-07T07:58:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f76a0066f30 0x7f76a00693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76bc162170 0x7f76bc155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76bc068250 0x7f76bc14ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 --2- 172.21.15.7:0/137886959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc068620 0x7f76bc161930 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 >> 172.21.15.7:0/137886959 conn(0x7f76bc0c9c30 msgr2=0x7f76bc156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 shutdown_connections 2023-10-07T07:58:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.169+0000 7f76c36d2700 1 -- 172.21.15.7:0/137886959 wait complete. 2023-10-07T07:58:37.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-07T07:58:37.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.314+0000 7f00ec131700 1 Processor -- start 2023-10-07T07:58:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.315+0000 7f00ec131700 1 -- start start 2023-10-07T07:58:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.315+0000 7f00ec131700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40d0250 0x7f00e40d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.315+0000 7f00ec131700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00e40b3b60 0x7f00e40b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.315+0000 7f00ec131700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e40b61f0 con 0x7f00e4159380 2023-10-07T07:58:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.315+0000 7f00ec131700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e40d0be0 con 0x7f00e40b3b60 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.315+0000 7f00ec131700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e40b4820 con 0x7f00e40d0250 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00ea6ce700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00e40b3b60 0x7f00e40b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00ea6ce700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00e40b3b60 0x7f00e40b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34248/0 (socket says 172.21.15.7:34248) 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00ea6ce700 1 -- 172.21.15.7:0/4249933940 learned_addr learned my addr 172.21.15.7:0/4249933940 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00e96cc700 1 --2- 172.21.15.7:0/4249933940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40d0250 0x7f00e40d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00ea6ce700 1 -- 172.21.15.7:0/4249933940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40d0250 msgr2=0x7f00e40d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00ea6ce700 1 --2- 172.21.15.7:0/4249933940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40d0250 0x7f00e40d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00ea6ce700 1 -- 172.21.15.7:0/4249933940 >> v1:172.21.15.172:6789/0 conn(0x7f00e4159380 legacy=0x7f00e40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00ea6ce700 1 -- 172.21.15.7:0/4249933940 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e40b5000 con 0x7f00e40b3b60 2023-10-07T07:58:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00e96cc700 1 --2- 172.21.15.7:0/4249933940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40d0250 0x7f00e40d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:37.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00ea6ce700 1 --2- 172.21.15.7:0/4249933940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40b3b60 0x7f00e40b3fe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009f20 tx=0x7f00dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d83886ed6fdd94 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:37.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00e8ecb700 1 -- 172.21.15.7:0/4249933940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00e070 con 0x7f00e40b3b60 2023-10-07T07:58:37.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00e8ecb700 1 -- 172.21.15.7:0/4249933940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc00ba80 con 0x7f00e40b3b60 2023-10-07T07:58:37.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.316+0000 7f00e8ecb700 1 -- 172.21.15.7:0/4249933940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0092e0 con 0x7f00e40b3b60 2023-10-07T07:58:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.317+0000 7f00ec131700 1 -- 172.21.15.7:0/4249933940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40b3b60 msgr2=0x7f00e40b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.317+0000 7f00ec131700 1 --2- 172.21.15.7:0/4249933940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40b3b60 0x7f00e40b3fe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009f20 tx=0x7f00dc009840 comp rx=0 tx=0).stop 2023-10-07T07:58:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.317+0000 7f00ec131700 1 -- 172.21.15.7:0/4249933940 shutdown_connections 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.317+0000 7f00ec131700 1 --2- 172.21.15.7:0/4249933940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40b3b60 0x7f00e40b3fe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.317+0000 7f00ec131700 1 --2- 172.21.15.7:0/4249933940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40d0250 0x7f00e40d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.317+0000 7f00ec131700 1 -- 172.21.15.7:0/4249933940 >> 172.21.15.7:0/4249933940 conn(0x7f00e4067be0 msgr2=0x7f00e4067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.317+0000 7f00ec131700 1 -- 172.21.15.7:0/4249933940 shutdown_connections 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.317+0000 7f00ec131700 1 -- 172.21.15.7:0/4249933940 wait complete. 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.318+0000 7f00ec131700 1 Processor -- start 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.318+0000 7f00ec131700 1 -- start start 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00ec131700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40b3b60 0x7f00e415c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00ec131700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40d0250 0x7f00e4157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00ec131700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4159380 0x7f00e4145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00ec131700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e4158090 con 0x7f00e40d0250 2023-10-07T07:58:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00ec131700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e415a410 con 0x7f00e4159380 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00ec131700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e4159ae0 con 0x7f00e40b3b60 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00e96cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40d0250 0x7f00e4157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00e96cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40d0250 0x7f00e4157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34256/0 (socket says 172.21.15.7:34256) 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00e96cc700 1 -- 172.21.15.7:0/3714391811 learned_addr learned my addr 172.21.15.7:0/3714391811 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00e9ecd700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40b3b60 0x7f00e415c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00ea6ce700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4159380 0x7f00e4145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00e96cc700 1 -- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40b3b60 msgr2=0x7f00e415c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00e96cc700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40b3b60 0x7f00e415c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00e96cc700 1 -- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4159380 msgr2=0x7f00e4145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00e96cc700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4159380 0x7f00e4145890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.319+0000 7f00e96cc700 1 -- 172.21.15.7:0/3714391811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc009ae0 con 0x7f00e40d0250 2023-10-07T07:58:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.320+0000 7f00e9ecd700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40b3b60 0x7f00e415c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.320+0000 7f00ea6ce700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4159380 0x7f00e4145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.320+0000 7f00e96cc700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40d0250 0x7f00e4157600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00d800e900 tx=0x7f00d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.320+0000 7f00d5ffb700 1 -- 172.21.15.7:0/3714391811 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d800eea0 con 0x7f00e40d0250 2023-10-07T07:58:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.320+0000 7f00d5ffb700 1 -- 172.21.15.7:0/3714391811 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d800c5e0 con 0x7f00e40d0250 2023-10-07T07:58:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.320+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e4159dc0 con 0x7f00e40d0250 2023-10-07T07:58:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.320+0000 7f00d5ffb700 1 -- 172.21.15.7:0/3714391811 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d800c980 con 0x7f00e40d0250 2023-10-07T07:58:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.320+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e4156900 con 0x7f00e40d0250 2023-10-07T07:58:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.321+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e415a5a0 con 0x7f00e40d0250 2023-10-07T07:58:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.321+0000 7f00d5ffb700 1 -- 172.21.15.7:0/3714391811 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00d800cd30 con 0x7f00e40d0250 2023-10-07T07:58:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.322+0000 7f00d5ffb700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f00c8066f30 0x7f00c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.322+0000 7f00d5ffb700 1 -- 172.21.15.7:0/3714391811 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f00d8083f80 con 0x7f00e40d0250 2023-10-07T07:58:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.322+0000 7f00e9ecd700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f00c8066f30 0x7f00c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.323+0000 7f00e9ecd700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f00c8066f30 0x7f00c80693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00e0000c00 tx=0x7f00e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.326+0000 7f00d5ffb700 1 -- 172.21.15.7:0/3714391811 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00d8054fd0 con 0x7f00e40d0250 2023-10-07T07:58:37.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.529+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f00e40716c0 con 0x7f00e40d0250 2023-10-07T07:58:37.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.529+0000 7f00d5ffb700 1 -- 172.21.15.7:0/3714391811 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f00d8054b60 con 0x7f00e40d0250 2023-10-07T07:58:37.531 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:37.531 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2023-10-07T07:58:37.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:37.531 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2023-10-07T07:58:37.531 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2023-10-07T07:58:37.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:37.531 INFO:teuthology.orchestra.run.smithi007.stdout:] 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f00c8066f30 msgr2=0x7f00c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f00c8066f30 0x7f00c80693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00e0000c00 tx=0x7f00e0009040 comp rx=0 tx=0).stop 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40d0250 msgr2=0x7f00e4157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40d0250 0x7f00e4157600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00d800e900 tx=0x7f00d8008040 comp rx=0 tx=0).stop 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 shutdown_connections 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f00c8066f30 0x7f00c80693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4159380 0x7f00e4145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00e40b3b60 0x7f00e415c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 --2- 172.21.15.7:0/3714391811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e40d0250 0x7f00e4157600 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 >> 172.21.15.7:0/3714391811 conn(0x7f00e4067be0 msgr2=0x7f00e40b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:37.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.531+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 shutdown_connections 2023-10-07T07:58:37.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.532+0000 7f00ec131700 1 -- 172.21.15.7:0/3714391811 wait complete. 2023-10-07T07:58:37.546 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-07T07:58:37.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-07T07:58:37.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.681+0000 7f91f23f1700 1 Processor -- start 2023-10-07T07:58:37.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.681+0000 7f91f23f1700 1 -- start start 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91f23f1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 0x7f91ec06af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91f23f1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f91ec0c1d30 0x7f91ec0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91f23f1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ec0a9920 con 0x7f91ec1594f0 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91f23f1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ec0c26f0 con 0x7f91ec0c1d30 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91f23f1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec06b4f0 con 0x7f91ec1598c0 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91ebfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91ec1594f0 0x7f91ec0674f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55210/0 (socket says 172.21.15.7:55210) 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91ebfff700 1 -- 172.21.15.7:0/3664917998 learned_addr learned my addr 172.21.15.7:0/3664917998 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91f098e700 1 --2- 172.21.15.7:0/3664917998 >> v2:172.21.15.172:3300/0 conn(0x7f91ec0c1d30 0x7f91ec0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91eb7fe700 1 --2- 172.21.15.7:0/3664917998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 0x7f91ec06af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91eaffd700 1 -- 172.21.15.7:0/3664917998 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651024567 0 0) 0x7f91ec0a9920 con 0x7f91ec1594f0 2023-10-07T07:58:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91eaffd700 1 -- 172.21.15.7:0/3664917998 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91d4003570 con 0x7f91ec1594f0 2023-10-07T07:58:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91eaffd700 1 -- 172.21.15.7:0/3664917998 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 710436874 0 0) 0x7f91d4003570 con 0x7f91ec1594f0 2023-10-07T07:58:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91f098e700 1 -- 172.21.15.7:0/3664917998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 msgr2=0x7f91ec06af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91f098e700 1 --2- 172.21.15.7:0/3664917998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 0x7f91ec06af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.682+0000 7f91f098e700 1 -- 172.21.15.7:0/3664917998 >> v1:172.21.15.7:6789/0 conn(0x7f91ec1594f0 legacy=0x7f91ec0674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.683+0000 7f91f098e700 1 -- 172.21.15.7:0/3664917998 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec06b7d0 con 0x7f91ec0c1d30 2023-10-07T07:58:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.683+0000 7f91f098e700 1 --2- 172.21.15.7:0/3664917998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ec0c1d30 0x7f91ec0c21b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009e60 tx=0x7f91e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d20c7a5f1db8c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.683+0000 7f91eaffd700 1 -- 172.21.15.7:0/3664917998 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e400e070 con 0x7f91ec0c1d30 2023-10-07T07:58:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.683+0000 7f91eaffd700 1 -- 172.21.15.7:0/3664917998 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e400b7a0 con 0x7f91ec0c1d30 2023-10-07T07:58:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.683+0000 7f91eaffd700 1 -- 172.21.15.7:0/3664917998 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc001470 con 0x7f91ec0c1d30 2023-10-07T07:58:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.683+0000 7f91f23f1700 1 -- 172.21.15.7:0/3664917998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ec0c1d30 msgr2=0x7f91ec0c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.683+0000 7f91f23f1700 1 --2- 172.21.15.7:0/3664917998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ec0c1d30 0x7f91ec0c21b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009e60 tx=0x7f91e4009780 comp rx=0 tx=0).stop 2023-10-07T07:58:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.684+0000 7f91f23f1700 1 -- 172.21.15.7:0/3664917998 shutdown_connections 2023-10-07T07:58:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.684+0000 7f91f23f1700 1 --2- 172.21.15.7:0/3664917998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ec0c1d30 0x7f91ec0c21b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.684+0000 7f91f23f1700 1 --2- 172.21.15.7:0/3664917998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 0x7f91ec06af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.684+0000 7f91f23f1700 1 -- 172.21.15.7:0/3664917998 >> 172.21.15.7:0/3664917998 conn(0x7f91ec0cda60 msgr2=0x7f91ec068100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.684+0000 7f91f23f1700 1 -- 172.21.15.7:0/3664917998 shutdown_connections 2023-10-07T07:58:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.684+0000 7f91f23f1700 1 -- 172.21.15.7:0/3664917998 wait complete. 2023-10-07T07:58:37.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.685+0000 7f91f23f1700 1 Processor -- start 2023-10-07T07:58:37.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.685+0000 7f91f23f1700 1 -- start start 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91f23f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec0c1d30 0x7f91ec0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91f23f1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ec1594f0 0x7f91ec1547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91f23f1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 0x7f91ec154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91f23f1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec155230 con 0x7f91ec0c1d30 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91f23f1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec0c5400 con 0x7f91ec1594f0 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91f23f1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec0c4af0 con 0x7f91ec1598c0 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec0c1d30 0x7f91ec0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec0c1d30 0x7f91ec0b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34262/0 (socket says 172.21.15.7:34262) 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91ebfff700 1 -- 172.21.15.7:0/876020620 learned_addr learned my addr 172.21.15.7:0/876020620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91eb7fe700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ec1594f0 0x7f91ec1547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91f098e700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 0x7f91ec154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91ebfff700 1 -- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 msgr2=0x7f91ec154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91ebfff700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 0x7f91ec154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91ebfff700 1 -- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ec1594f0 msgr2=0x7f91ec1547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91ebfff700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ec1594f0 0x7f91ec1547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91ebfff700 1 -- 172.21.15.7:0/876020620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e4009a20 con 0x7f91ec0c1d30 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.686+0000 7f91ebfff700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec0c1d30 0x7f91ec0b1680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00cb90 tx=0x7f91dc001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.687+0000 7f91d3fff700 1 -- 172.21.15.7:0/876020620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc007e90 con 0x7f91ec0c1d30 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.687+0000 7f91d3fff700 1 -- 172.21.15.7:0/876020620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc007360 con 0x7f91ec0c1d30 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.687+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec0c4db0 con 0x7f91ec0c1d30 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.687+0000 7f91d3fff700 1 -- 172.21.15.7:0/876020620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc007700 con 0x7f91ec0c1d30 2023-10-07T07:58:37.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.687+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec0c5890 con 0x7f91ec0c1d30 2023-10-07T07:58:37.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.688+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ec0c56e0 con 0x7f91ec0c1d30 2023-10-07T07:58:37.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.688+0000 7f91d3fff700 1 -- 172.21.15.7:0/876020620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91dc004a40 con 0x7f91ec0c1d30 2023-10-07T07:58:37.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.689+0000 7f91d3fff700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f91d8066f30 0x7f91d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:37.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.689+0000 7f91d3fff700 1 -- 172.21.15.7:0/876020620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f91dc0839a0 con 0x7f91ec0c1d30 2023-10-07T07:58:37.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.689+0000 7f91eb7fe700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f91d8066f30 0x7f91d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:37.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.689+0000 7f91eb7fe700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f91d8066f30 0x7f91d80693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f91ec15b4f0 tx=0x7f91e00091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:37.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.693+0000 7f91d3fff700 1 -- 172.21.15.7:0/876020620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91dc050fe0 con 0x7f91ec0c1d30 2023-10-07T07:58:37.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:37.896+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f91ec0716c0 con 0x7f91ec0c1d30 2023-10-07T07:58:38.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.562+0000 7f91d3fff700 1 -- 172.21.15.7:0/876020620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f91dc018020 con 0x7f91ec0c1d30 2023-10-07T07:58:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f91d8066f30 msgr2=0x7f91d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f91d8066f30 0x7f91d80693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f91ec15b4f0 tx=0x7f91e00091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec0c1d30 msgr2=0x7f91ec0b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec0c1d30 0x7f91ec0b1680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00cb90 tx=0x7f91dc001b50 comp rx=0 tx=0).stop 2023-10-07T07:58:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 shutdown_connections 2023-10-07T07:58:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f91d8066f30 0x7f91d80693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ec1598c0 0x7f91ec154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec0c1d30 0x7f91ec0b1680 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 --2- 172.21.15.7:0/876020620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ec1594f0 0x7f91ec1547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 >> 172.21.15.7:0/876020620 conn(0x7f91ec0cda60 msgr2=0x7f91ec0680e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.566+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 shutdown_connections 2023-10-07T07:58:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.567+0000 7f91f23f1700 1 -- 172.21.15.7:0/876020620 wait complete. 2023-10-07T07:58:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2023-10-07T07:58:38.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-07T07:58:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.727+0000 7feb4404d700 1 Processor -- start 2023-10-07T07:58:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.727+0000 7feb4404d700 1 -- start start 2023-10-07T07:58:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.728+0000 7feb4404d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb3c0cc2b0 0x7feb3c068000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.728+0000 7feb4404d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c0689a0 con 0x7feb3c068540 2023-10-07T07:58:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.728+0000 7feb4404d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c0c8360 con 0x7feb3c0cbee0 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.728+0000 7feb4404d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3c0c7480 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.728+0000 7feb425ea700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7feb3c068540 0x7feb3c0ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55218/0 (socket says 172.21.15.7:55218) 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.728+0000 7feb425ea700 1 -- 172.21.15.7:0/2103747460 learned_addr learned my addr 172.21.15.7:0/2103747460 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.728+0000 7feb415e8700 1 --2- 172.21.15.7:0/2103747460 >> v2:172.21.15.172:3300/0 conn(0x7feb3c0cc2b0 0x7feb3c068000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.729+0000 7feb40de7700 1 -- 172.21.15.7:0/2103747460 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455633437 0 0) 0x7feb34001490 con 0x7feb3c068540 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.729+0000 7feb40de7700 1 -- 172.21.15.7:0/2103747460 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb28003570 con 0x7feb3c068540 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.729+0000 7feb40de7700 1 -- 172.21.15.7:0/2103747460 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764223026 0 0) 0x7feb3c0c8360 con 0x7feb3c0cbee0 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.729+0000 7feb40de7700 1 -- 172.21.15.7:0/2103747460 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb34001490 con 0x7feb3c0cbee0 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.729+0000 7feb415e8700 1 -- 172.21.15.7:0/2103747460 >> v1:172.21.15.172:6789/0 conn(0x7feb3c0cbee0 legacy=0x7feb3c0c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.729+0000 7feb415e8700 1 -- 172.21.15.7:0/2103747460 >> v1:172.21.15.7:6789/0 conn(0x7feb3c068540 legacy=0x7feb3c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.729+0000 7feb40de7700 1 -- 172.21.15.7:0/2103747460 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3497817702 0 0) 0x7feb28003570 con 0x7feb3c068540 2023-10-07T07:58:38.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.729+0000 7feb415e8700 1 -- 172.21.15.7:0/2103747460 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c0c7750 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.729+0000 7feb415e8700 1 --2- 172.21.15.7:0/2103747460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c0cc2b0 0x7feb3c068000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0098e0 tx=0x7feb2c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891a872db4d9b3d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:38.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb40de7700 1 -- 172.21.15.7:0/2103747460 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34001980 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb40de7700 1 -- 172.21.15.7:0/2103747460 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c011420 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb40de7700 1 -- 172.21.15.7:0/2103747460 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0117c0 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb4404d700 1 -- 172.21.15.7:0/2103747460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c0cc2b0 msgr2=0x7feb3c068000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb4404d700 1 --2- 172.21.15.7:0/2103747460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c0cc2b0 0x7feb3c068000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0098e0 tx=0x7feb2c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb4404d700 1 -- 172.21.15.7:0/2103747460 shutdown_connections 2023-10-07T07:58:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb4404d700 1 --2- 172.21.15.7:0/2103747460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c0cc2b0 0x7feb3c068000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb4404d700 1 -- 172.21.15.7:0/2103747460 >> 172.21.15.7:0/2103747460 conn(0x7feb3c0c9c30 msgr2=0x7feb3c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb4404d700 1 -- 172.21.15.7:0/2103747460 shutdown_connections 2023-10-07T07:58:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.730+0000 7feb4404d700 1 -- 172.21.15.7:0/2103747460 wait complete. 2023-10-07T07:58:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.732+0000 7feb4404d700 1 Processor -- start 2023-10-07T07:58:38.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.732+0000 7feb4404d700 1 -- start start 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.732+0000 7feb4404d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c068540 0x7feb3c150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.732+0000 7feb4404d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3c0cbee0 0x7feb3c14e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.732+0000 7feb4404d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb3c0cc2b0 0x7feb3c14cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.732+0000 7feb4404d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c14e2d0 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.732+0000 7feb4404d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c14d6c0 con 0x7feb3c068540 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.732+0000 7feb4404d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c14d9a0 con 0x7feb3c0cbee0 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb425ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb3c0cc2b0 0x7feb3c14cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb425ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb3c0cc2b0 0x7feb3c14cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34276/0 (socket says 172.21.15.7:34276) 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb425ea700 1 -- 172.21.15.7:0/853525629 learned_addr learned my addr 172.21.15.7:0/853525629 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb41de9700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c068540 0x7feb3c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb415e8700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3c0cbee0 0x7feb3c14e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:38.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb425ea700 1 -- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3c0cbee0 msgr2=0x7feb3c14e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb425ea700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3c0cbee0 0x7feb3c14e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb425ea700 1 -- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c068540 msgr2=0x7feb3c150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb425ea700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c068540 0x7feb3c150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb425ea700 1 -- 172.21.15.7:0/853525629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c009420 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb425ea700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb3c0cc2b0 0x7feb3c14cfc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb3400cdc0 tx=0x7feb34001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb31ffb700 1 -- 172.21.15.7:0/853525629 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34004dc0 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb31ffb700 1 -- 172.21.15.7:0/853525629 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3400a870 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c14dc80 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.733+0000 7feb31ffb700 1 -- 172.21.15.7:0/853525629 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400ac30 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.734+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c14bd50 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.735+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3c04c3c0 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.735+0000 7feb31ffb700 1 -- 172.21.15.7:0/853525629 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb34007360 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.736+0000 7feb31ffb700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7feb20066f30 0x7feb200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.736+0000 7feb31ffb700 1 -- 172.21.15.7:0/853525629 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5223+0+0 (secure 0 0 0) 0x7feb34083cd0 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.736+0000 7feb41de9700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7feb20066f30 0x7feb200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:38.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.737+0000 7feb41de9700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7feb20066f30 0x7feb200693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb3c151a00 tx=0x7feb38007d40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:38.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.741+0000 7feb31ffb700 1 -- 172.21.15.7:0/853525629 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb340511a0 con 0x7feb3c0cc2b0 2023-10-07T07:58:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:38.945+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7feb3c0689a0 con 0x7feb3c0cc2b0 2023-10-07T07:58:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.570+0000 7feb31ffb700 1 -- 172.21.15.7:0/853525629 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7feb3c0689a0 con 0x7feb3c0cc2b0 2023-10-07T07:58:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.576+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7feb20066f30 msgr2=0x7feb200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.577+0000 7feb4404d700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7feb20066f30 0x7feb200693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb3c151a00 tx=0x7feb38007d40 comp rx=0 tx=0).stop 2023-10-07T07:58:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.577+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb3c0cc2b0 msgr2=0x7feb3c14cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.577+0000 7feb4404d700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb3c0cc2b0 0x7feb3c14cfc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb3400cdc0 tx=0x7feb34001fe0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.578+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 shutdown_connections 2023-10-07T07:58:39.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.578+0000 7feb4404d700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7feb20066f30 0x7feb200693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.578+0000 7feb4404d700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb3c0cc2b0 0x7feb3c14cfc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.578+0000 7feb4404d700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c068540 0x7feb3c150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.578+0000 7feb4404d700 1 --2- 172.21.15.7:0/853525629 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3c0cbee0 0x7feb3c14e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.578+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 >> 172.21.15.7:0/853525629 conn(0x7feb3c0c9c30 msgr2=0x7feb3c0c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:39.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.580+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 shutdown_connections 2023-10-07T07:58:39.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.581+0000 7feb4404d700 1 -- 172.21.15.7:0/853525629 wait complete. 2023-10-07T07:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2023-10-07T07:58:39.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-07T07:58:39.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.742+0000 7f4c6993c700 1 Processor -- start 2023-10-07T07:58:39.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.742+0000 7f4c6993c700 1 -- start start 2023-10-07T07:58:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.743+0000 7f4c6993c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c64157300 0x7f4c6406e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.743+0000 7f4c6993c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c640c8020 con 0x7f4c64067610 2023-10-07T07:58:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.743+0000 7f4c6993c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6406f3b0 con 0x7f4c64067240 2023-10-07T07:58:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.743+0000 7f4c6993c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640c76f0 con 0x7f4c64157300 2023-10-07T07:58:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.743+0000 7f4c637fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4c64067610 0x7f4c6406eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58116/0 (socket says 172.21.15.7:58116) 2023-10-07T07:58:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.743+0000 7f4c637fe700 1 -- 172.21.15.7:0/3591494652 learned_addr learned my addr 172.21.15.7:0/3591494652 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c5a7fc700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c64157300 0x7f4c6406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705230698 0 0) 0x7f4c640c8020 con 0x7f4c64067610 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c50003570 con 0x7f4c64067610 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3624608220 0 0) 0x7f4c50003570 con 0x7f4c64067610 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c64157300 msgr2=0x7f4c6406e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c64157300 0x7f4c6406e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 >> v1:172.21.15.172:6789/0 conn(0x7f4c64067240 legacy=0x7f4c64156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c640c79d0 con 0x7f4c64067610 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f4c5c0017f0 con 0x7f4c64067610 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 >> v1:172.21.15.7:6789/0 conn(0x7f4c64067610 legacy=0x7f4c6406eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c50003a90 0x7f4c50003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c64157300 0x7f4c500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640c76f0 con 0x7f4c50004810 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5000bb10 con 0x7f4c50003a90 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5000bcd0 con 0x7f4c64157300 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c5a7fc700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c62ffd700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c50003a90 0x7f4c50003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.744+0000 7f4c637fe700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c64157300 0x7f4c500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.745+0000 7f4c5a7fc700 1 -- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c64157300 msgr2=0x7f4c500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.745+0000 7f4c5a7fc700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c64157300 0x7f4c500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.745+0000 7f4c5a7fc700 1 -- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c50003a90 msgr2=0x7f4c50003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.745+0000 7f4c5a7fc700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c50003a90 0x7f4c50003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.745+0000 7f4c5a7fc700 1 -- 172.21.15.7:0/3591494652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c50003570 con 0x7f4c50004810 2023-10-07T07:58:39.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.745+0000 7f4c5a7fc700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c50004c80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009c00 tx=0x7f4c4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ea71d1b005dfc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.745+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00e710 con 0x7f4c50004810 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.745+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c00e870 con 0x7f4c50004810 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.745+0000 7f4c627fc700 1 -- 172.21.15.7:0/3591494652 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00ec30 con 0x7f4c50004810 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.746+0000 7f4c6993c700 1 -- 172.21.15.7:0/3591494652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 msgr2=0x7f4c50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.746+0000 7f4c6993c700 1 --2- 172.21.15.7:0/3591494652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c50004c80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009c00 tx=0x7f4c4c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.746+0000 7f4c637fe700 1 -- 172.21.15.7:0/3591494652 reap_dead start 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.746+0000 7f4c6993c700 1 -- 172.21.15.7:0/3591494652 shutdown_connections 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.746+0000 7f4c6993c700 1 -- 172.21.15.7:0/3591494652 >> 172.21.15.7:0/3591494652 conn(0x7f4c64068100 msgr2=0x7f4c64068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.746+0000 7f4c6993c700 1 -- 172.21.15.7:0/3591494652 shutdown_connections 2023-10-07T07:58:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.746+0000 7f4c6993c700 1 -- 172.21.15.7:0/3591494652 wait complete. 2023-10-07T07:58:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.747+0000 7f4c6993c700 1 Processor -- start 2023-10-07T07:58:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.747+0000 7f4c6993c700 1 -- start start 2023-10-07T07:58:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c6993c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c640af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c6993c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6414ea70 0x7f4c640ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c6993c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c6414eee0 0x7f4c640acc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c6993c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640ad180 con 0x7f4c50004810 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c6993c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c64150e10 con 0x7f4c6414eee0 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c6993c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640a99f0 con 0x7f4c6414ea70 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c62ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c640af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c62ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c640af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56248/0 (socket says 172.21.15.7:56248) 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c62ffd700 1 -- 172.21.15.7:0/1064224451 learned_addr learned my addr 172.21.15.7:0/1064224451 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c637fe700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c6414eee0 0x7f4c640acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c5a7fc700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6414ea70 0x7f4c640ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c62ffd700 1 -- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6414ea70 msgr2=0x7f4c640ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c62ffd700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6414ea70 0x7f4c640ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c62ffd700 1 -- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c6414eee0 msgr2=0x7f4c640acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c62ffd700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c6414eee0 0x7f4c640acc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c62ffd700 1 -- 172.21.15.7:0/1064224451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c640c4050 con 0x7f4c50004810 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.748+0000 7f4c5a7fc700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6414ea70 0x7f4c640ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.749+0000 7f4c62ffd700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c640af080 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400f950 tx=0x7f4c54007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.749+0000 7f4c5b7fe700 1 -- 172.21.15.7:0/1064224451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400d670 con 0x7f4c50004810 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.749+0000 7f4c5b7fe700 1 -- 172.21.15.7:0/1064224451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5400d7f0 con 0x7f4c50004810 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.749+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c640c4330 con 0x7f4c50004810 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.749+0000 7f4c5b7fe700 1 -- 172.21.15.7:0/1064224451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400dbb0 con 0x7f4c50004810 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.749+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c640c48f0 con 0x7f4c50004810 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.750+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6404c3c0 con 0x7f4c50004810 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.750+0000 7f4c5b7fe700 1 -- 172.21.15.7:0/1064224451 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c5401c400 con 0x7f4c50004810 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.751+0000 7f4c5b7fe700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c44066f30 0x7f4c440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.751+0000 7f4c5b7fe700 1 -- 172.21.15.7:0/1064224451 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f4c540849a0 con 0x7f4c50004810 2023-10-07T07:58:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.751+0000 7f4c5a7fc700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c44066f30 0x7f4c440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.751+0000 7f4c5a7fc700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c44066f30 0x7f4c440693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0091e0 tx=0x7f4c4c01a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:39.755+0000 7f4c5b7fe700 1 -- 172.21.15.7:0/1064224451 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c54051c80 con 0x7f4c50004810 2023-10-07T07:58:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.001+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4c640c7cb0 con 0x7f4c50004810 2023-10-07T07:58:40.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.585+0000 7f4c5b7fe700 1 -- 172.21.15.7:0/1064224451 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4c54017020 con 0x7f4c50004810 2023-10-07T07:58:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.589+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c44066f30 msgr2=0x7f4c440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:40.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.589+0000 7f4c6993c700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c44066f30 0x7f4c440693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0091e0 tx=0x7f4c4c01a000 comp rx=0 tx=0).stop 2023-10-07T07:58:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.589+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 msgr2=0x7f4c640af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:40.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.589+0000 7f4c6993c700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c640af080 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400f950 tx=0x7f4c54007d30 comp rx=0 tx=0).stop 2023-10-07T07:58:40.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.592+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 shutdown_connections 2023-10-07T07:58:40.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.592+0000 7f4c6993c700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c44066f30 0x7f4c440693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.592+0000 7f4c6993c700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c6414eee0 0x7f4c640acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.592+0000 7f4c6993c700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c50004810 0x7f4c640af080 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.592+0000 7f4c6993c700 1 --2- 172.21.15.7:0/1064224451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6414ea70 0x7f4c640ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.592+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 >> 172.21.15.7:0/1064224451 conn(0x7f4c64068100 msgr2=0x7f4c640afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:40.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.592+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 shutdown_connections 2023-10-07T07:58:40.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.592+0000 7f4c6993c700 1 -- 172.21.15.7:0/1064224451 wait complete. 2023-10-07T07:58:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 4 and data pool 5 2023-10-07T07:58:40.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:40.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.752+0000 7f1d93027700 1 Processor -- start 2023-10-07T07:58:40.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.752+0000 7f1d93027700 1 -- start start 2023-10-07T07:58:40.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.753+0000 7f1d93027700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1d8c1570a0 0x7f1d8c0c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:40.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.753+0000 7f1d93027700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d8c06bd30 0x7f1d8c0c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:40.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.753+0000 7f1d93027700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8c0c3b80 con 0x7f1d8c157470 2023-10-07T07:58:40.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.753+0000 7f1d93027700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c0c2f70 con 0x7f1d8c06bd30 2023-10-07T07:58:40.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.753+0000 7f1d93027700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c0c3250 con 0x7f1d8c1570a0 2023-10-07T07:58:40.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.753+0000 7f1d915c4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d8c06bd30 0x7f1d8c0c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:40.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d915c4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d8c06bd30 0x7f1d8c0c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56260/0 (socket says 172.21.15.7:56260) 2023-10-07T07:58:40.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d915c4700 1 -- 172.21.15.7:0/1577943073 learned_addr learned my addr 172.21.15.7:0/1577943073 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d90dc3700 1 --2- 172.21.15.7:0/1577943073 >> v2:172.21.15.172:3300/0 conn(0x7f1d8c1570a0 0x7f1d8c0c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d915c4700 1 -- 172.21.15.7:0/1577943073 >> v2:172.21.15.172:3300/0 conn(0x7f1d8c1570a0 msgr2=0x7f1d8c0c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d915c4700 1 --2- 172.21.15.7:0/1577943073 >> v2:172.21.15.172:3300/0 conn(0x7f1d8c1570a0 0x7f1d8c0c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d915c4700 1 -- 172.21.15.7:0/1577943073 >> v1:172.21.15.172:6789/0 conn(0x7f1d8c157470 legacy=0x7f1d8c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d915c4700 1 -- 172.21.15.7:0/1577943073 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0c3530 con 0x7f1d8c06bd30 2023-10-07T07:58:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d90dc3700 1 --2- 172.21.15.7:0/1577943073 >> v2:172.21.15.172:3300/0 conn(0x7f1d8c1570a0 0x7f1d8c0c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d915c4700 1 --2- 172.21.15.7:0/1577943073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c06bd30 0x7f1d8c0c4010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0caa70 tx=0x7f1d84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6049982a7e252526 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/1577943073 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400e070 con 0x7f1d8c06bd30 2023-10-07T07:58:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/1577943073 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8400ba50 con 0x7f1d8c06bd30 2023-10-07T07:58:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.754+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/1577943073 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840092e0 con 0x7f1d8c06bd30 2023-10-07T07:58:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.755+0000 7f1d93027700 1 -- 172.21.15.7:0/1577943073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c06bd30 msgr2=0x7f1d8c0c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.755+0000 7f1d93027700 1 --2- 172.21.15.7:0/1577943073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c06bd30 0x7f1d8c0c4010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0caa70 tx=0x7f1d84009840 comp rx=0 tx=0).stop 2023-10-07T07:58:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.755+0000 7f1d93027700 1 -- 172.21.15.7:0/1577943073 shutdown_connections 2023-10-07T07:58:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.755+0000 7f1d93027700 1 --2- 172.21.15.7:0/1577943073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c06bd30 0x7f1d8c0c4010 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.755+0000 7f1d93027700 1 --2- 172.21.15.7:0/1577943073 >> v2:172.21.15.172:3300/0 conn(0x7f1d8c1570a0 0x7f1d8c0c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.755+0000 7f1d93027700 1 -- 172.21.15.7:0/1577943073 >> 172.21.15.7:0/1577943073 conn(0x7f1d8c067ac0 msgr2=0x7f1d8c0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.755+0000 7f1d93027700 1 -- 172.21.15.7:0/1577943073 shutdown_connections 2023-10-07T07:58:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.755+0000 7f1d93027700 1 -- 172.21.15.7:0/1577943073 wait complete. 2023-10-07T07:58:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.756+0000 7f1d93027700 1 Processor -- start 2023-10-07T07:58:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.756+0000 7f1d93027700 1 -- start start 2023-10-07T07:58:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.757+0000 7f1d93027700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8c06bd30 0x7f1d8c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.757+0000 7f1d93027700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1570a0 0x7f1d8c16b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.757+0000 7f1d93027700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d8c157470 0x7f1d8c154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.757+0000 7f1d93027700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c16be50 con 0x7f1d8c1570a0 2023-10-07T07:58:40.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.757+0000 7f1d93027700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c167a60 con 0x7f1d8c157470 2023-10-07T07:58:40.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.757+0000 7f1d93027700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c153940 con 0x7f1d8c06bd30 2023-10-07T07:58:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.757+0000 7f1d8bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1570a0 0x7f1d8c16b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d8bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1570a0 0x7f1d8c16b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56274/0 (socket says 172.21.15.7:56274) 2023-10-07T07:58:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d8bfff700 1 -- 172.21.15.7:0/599444158 learned_addr learned my addr 172.21.15.7:0/599444158 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d90dc3700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8c06bd30 0x7f1d8c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d915c4700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d8c157470 0x7f1d8c154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d8bfff700 1 -- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8c06bd30 msgr2=0x7f1d8c153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d8bfff700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8c06bd30 0x7f1d8c153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d8bfff700 1 -- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d8c157470 msgr2=0x7f1d8c154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d8bfff700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d8c157470 0x7f1d8c154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d8bfff700 1 -- 172.21.15.7:0/599444158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009ae0 con 0x7f1d8c1570a0 2023-10-07T07:58:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d8bfff700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1570a0 0x7f1d8c16b910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c5c0 tx=0x7f1d7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d88ff9700 1 -- 172.21.15.7:0/599444158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78011590 con 0x7f1d8c1570a0 2023-10-07T07:58:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d88ff9700 1 -- 172.21.15.7:0/599444158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78011710 con 0x7f1d8c1570a0 2023-10-07T07:58:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d88ff9700 1 -- 172.21.15.7:0/599444158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78011ad0 con 0x7f1d8c1570a0 2023-10-07T07:58:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8c153c20 con 0x7f1d8c1570a0 2023-10-07T07:58:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.758+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c1541e0 con 0x7f1d8c1570a0 2023-10-07T07:58:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.759+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c167c30 con 0x7f1d8c1570a0 2023-10-07T07:58:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.760+0000 7f1d88ff9700 1 -- 172.21.15.7:0/599444158 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d7801b3b0 con 0x7f1d8c1570a0 2023-10-07T07:58:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.761+0000 7f1d88ff9700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1d70066f30 0x7f1d700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.761+0000 7f1d88ff9700 1 -- 172.21.15.7:0/599444158 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d78083c70 con 0x7f1d8c1570a0 2023-10-07T07:58:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.761+0000 7f1d90dc3700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1d70066f30 0x7f1d700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.762+0000 7f1d90dc3700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1d70066f30 0x7f1d700693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d80006f00 tx=0x7f1d80008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.765+0000 7f1d88ff9700 1 -- 172.21.15.7:0/599444158 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d78050ec0 con 0x7f1d8c1570a0 2023-10-07T07:58:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.969+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d8c167c30 con 0x7f1d8c1570a0 2023-10-07T07:58:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.970+0000 7f1d88ff9700 1 -- 172.21.15.7:0/599444158 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f1d8c167c30 con 0x7f1d8c1570a0 2023-10-07T07:58:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:40.972 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:58:40.003340+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:58:37.928486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T07:58:38.977756+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1d70066f30 msgr2=0x7f1d700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1d70066f30 0x7f1d700693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d80006f00 tx=0x7f1d80008040 comp rx=0 tx=0).stop 2023-10-07T07:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1570a0 msgr2=0x7f1d8c16b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1570a0 0x7f1d8c16b910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c5c0 tx=0x7f1d7800c8d0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 shutdown_connections 2023-10-07T07:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1d70066f30 0x7f1d700693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d8c157470 0x7f1d8c154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d8c06bd30 0x7f1d8c153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 --2- 172.21.15.7:0/599444158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d8c1570a0 0x7f1d8c16b910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 >> 172.21.15.7:0/599444158 conn(0x7f1d8c067ac0 msgr2=0x7f1d8c139c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 shutdown_connections 2023-10-07T07:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:40.973+0000 7f1d93027700 1 -- 172.21.15.7:0/599444158 wait complete. 2023-10-07T07:58:40.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:41.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.119+0000 7fb5075c8700 1 Processor -- start 2023-10-07T07:58:41.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.119+0000 7fb5075c8700 1 -- start start 2023-10-07T07:58:41.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb5075c8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb5001592c0 0x7fb50006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb5075c8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50006bb70 con 0x7fb500159690 2023-10-07T07:58:41.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb5075c8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5000cda10 con 0x7fb5000cb290 2023-10-07T07:58:41.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb5075c8700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50006af60 con 0x7fb5001592c0 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb504b63700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb500159690 0x7fb5000cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58126/0 (socket says 172.21.15.7:58126) 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb504b63700 1 -- 172.21.15.7:0/2297122679 learned_addr learned my addr 172.21.15.7:0/2297122679 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb505364700 1 --2- 172.21.15.7:0/2297122679 >> v2:172.21.15.172:3300/0 conn(0x7fb5001592c0 0x7fb50006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459820061 0 0) 0x7fb50006bb70 con 0x7fb500159690 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ec003570 con 0x7fb500159690 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020751834 0 0) 0x7fb5000cda10 con 0x7fb5000cb290 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb50006bb70 con 0x7fb5000cb290 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.120+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1275870913 0 0) 0x7fb4ec003570 con 0x7fb500159690 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 >> v2:172.21.15.172:3300/0 conn(0x7fb5001592c0 msgr2=0x7fb50006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 --2- 172.21.15.7:0/2297122679 >> v2:172.21.15.172:3300/0 conn(0x7fb5001592c0 0x7fb50006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 >> v1:172.21.15.172:6789/0 conn(0x7fb5000cb290 legacy=0x7fb5000a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50006b240 con 0x7fb500159690 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7fb4f0001470 con 0x7fb500159690 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb505364700 1 --2- 172.21.15.7:0/2297122679 >> v2:172.21.15.172:3300/0 conn(0x7fb5001592c0 0x7fb50006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 >> v1:172.21.15.7:6789/0 conn(0x7fb500159690 legacy=0x7fb5000cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4ec003bf0 0x7fb4ec003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb4ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4ec0051c0 0x7fb4ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50006af60 con 0x7fb4ec004810 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec00c0b0 con 0x7fb4ec003bf0 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec00c270 con 0x7fb4ec0051c0 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb504b63700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb4ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb505364700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4ec003bf0 0x7fb4ec003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb505b65700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4ec0051c0 0x7fb4ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb504b63700 1 -- 172.21.15.7:0/2297122679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4ec0051c0 msgr2=0x7fb4ec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb504b63700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4ec0051c0 0x7fb4ec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb504b63700 1 -- 172.21.15.7:0/2297122679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4ec003bf0 msgr2=0x7fb4ec003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb504b63700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4ec003bf0 0x7fb4ec003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb504b63700 1 -- 172.21.15.7:0/2297122679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec003570 con 0x7fb4ec004810 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb505b65700 1 -- 172.21.15.7:0/2297122679 reap_dead start 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.121+0000 7fb504b63700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb4ec004c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000c2d0 tx=0x7fb5000cdbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5c2501d9931752 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000cc00 con 0x7fb4ec004810 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f000cda0 con 0x7fb4ec004810 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb4f7fff700 1 -- 172.21.15.7:0/2297122679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0005b70 con 0x7fb4ec004810 2023-10-07T07:58:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb5075c8700 1 -- 172.21.15.7:0/2297122679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 msgr2=0x7fb4ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb5075c8700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb4ec004c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000c2d0 tx=0x7fb5000cdbe0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb5075c8700 1 -- 172.21.15.7:0/2297122679 shutdown_connections 2023-10-07T07:58:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb5075c8700 1 --2- 172.21.15.7:0/2297122679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb4ec004c80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb5075c8700 1 -- 172.21.15.7:0/2297122679 >> 172.21.15.7:0/2297122679 conn(0x7fb500068bd0 msgr2=0x7fb5000ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb5075c8700 1 -- 172.21.15.7:0/2297122679 shutdown_connections 2023-10-07T07:58:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.122+0000 7fb5075c8700 1 -- 172.21.15.7:0/2297122679 wait complete. 2023-10-07T07:58:41.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.124+0000 7fb5075c8700 1 Processor -- start 2023-10-07T07:58:41.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.124+0000 7fb5075c8700 1 -- start start 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb5075c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb500156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb5075c8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500149b10 0x7fb50015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb5075c8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb50015b650 0x7fb50015f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb5075c8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50015e5d0 con 0x7fb4ec004810 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb5075c8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50015e890 con 0x7fb500149b10 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb5075c8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50015eb70 con 0x7fb50015b650 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb505364700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb500156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb505364700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb500156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56292/0 (socket says 172.21.15.7:56292) 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb505364700 1 -- 172.21.15.7:0/1499106018 learned_addr learned my addr 172.21.15.7:0/1499106018 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb504b63700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500149b10 0x7fb50015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.125+0000 7fb505b65700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb50015b650 0x7fb50015f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb505364700 1 -- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb50015b650 msgr2=0x7fb50015f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb505364700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb50015b650 0x7fb50015f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb505364700 1 -- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500149b10 msgr2=0x7fb50015b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb505364700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500149b10 0x7fb50015b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb505364700 1 -- 172.21.15.7:0/1499106018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50015ee50 con 0x7fb4ec004810 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb505364700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb500156c30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc011d20 tx=0x7fb4fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb4f57fa700 1 -- 172.21.15.7:0/1499106018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00fac0 con 0x7fb4ec004810 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb4f57fa700 1 -- 172.21.15.7:0/1499106018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc00fc20 con 0x7fb4ec004810 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50015f130 con 0x7fb4ec004810 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb4f57fa700 1 -- 172.21.15.7:0/1499106018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc014440 con 0x7fb4ec004810 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.126+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5001573d0 con 0x7fb4ec004810 2023-10-07T07:58:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.127+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50015f370 con 0x7fb4ec004810 2023-10-07T07:58:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.128+0000 7fb4f57fa700 1 -- 172.21.15.7:0/1499106018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4fc027480 con 0x7fb4ec004810 2023-10-07T07:58:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.128+0000 7fb4f57fa700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb4e4066f60 0x7fb4e4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.128+0000 7fb4f57fa700 1 -- 172.21.15.7:0/1499106018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb4fc018030 con 0x7fb4ec004810 2023-10-07T07:58:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.129+0000 7fb504b63700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb4e4066f60 0x7fb4e4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.129+0000 7fb504b63700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb4e4066f60 0x7fb4e4069410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0007c90 tx=0x7fb4f0015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.132+0000 7fb4f57fa700 1 -- 172.21.15.7:0/1499106018 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4fc056190 con 0x7fb4ec004810 2023-10-07T07:58:41.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.333+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb50015f370 con 0x7fb4ec004810 2023-10-07T07:58:41.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.334+0000 7fb4f57fa700 1 -- 172.21.15.7:0/1499106018 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fb50015f370 con 0x7fb4ec004810 2023-10-07T07:58:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:41.335 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:58:40.003340+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:58:37.928486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T07:58:38.977756+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:41.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb4e4066f60 msgr2=0x7fb4e4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb4e4066f60 0x7fb4e4069410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0007c90 tx=0x7fb4f0015000 comp rx=0 tx=0).stop 2023-10-07T07:58:41.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 msgr2=0x7fb500156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb500156c30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc011d20 tx=0x7fb4fc0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 shutdown_connections 2023-10-07T07:58:41.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb4e4066f60 0x7fb4e4069410 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb50015b650 0x7fb50015f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ec004810 0x7fb500156c30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 --2- 172.21.15.7:0/1499106018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500149b10 0x7fb50015b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 >> 172.21.15.7:0/1499106018 conn(0x7fb500068bd0 msgr2=0x7fb5000c9970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:41.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 shutdown_connections 2023-10-07T07:58:41.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.336+0000 7fb5075c8700 1 -- 172.21.15.7:0/1499106018 wait complete. 2023-10-07T07:58:41.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-07T07:58:41.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.485+0000 7f6f26b0d700 1 Processor -- start 2023-10-07T07:58:41.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.485+0000 7f6f26b0d700 1 -- start start 2023-10-07T07:58:41.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.486+0000 7f6f26b0d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6f20156160 0x7f6f2006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.486+0000 7f6f26b0d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f2006e900 0x7f6f2006c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.486+0000 7f6f26b0d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2006b6f0 con 0x7f6f20155d90 2023-10-07T07:58:41.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.486+0000 7f6f26b0d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f200c7cf0 con 0x7f6f20156160 2023-10-07T07:58:41.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.486+0000 7f6f26b0d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2006ed60 con 0x7f6f2006e900 2023-10-07T07:58:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.486+0000 7f6f250aa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f2006e900 0x7f6f2006c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.486+0000 7f6f1ffff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6f20156160 0x7f6f2006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.486+0000 7f6f248a9700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6f20155d90 0x7f6f200c75e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:45516/0 (socket says 172.21.15.7:45516) 2023-10-07T07:58:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f248a9700 1 -- 172.21.15.7:0/3694433333 learned_addr learned my addr 172.21.15.7:0/3694433333 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f1ffff700 1 -- 172.21.15.7:0/3694433333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f2006e900 msgr2=0x7f6f2006c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f1ffff700 1 --2- 172.21.15.7:0/3694433333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f2006e900 0x7f6f2006c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f1ffff700 1 -- 172.21.15.7:0/3694433333 >> v1:172.21.15.172:6789/0 conn(0x7f6f20155d90 legacy=0x7f6f200c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f1f7fe700 1 -- 172.21.15.7:0/3694433333 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210770071 0 0) 0x7f6f2006b6f0 con 0x7f6f20155d90 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f1ffff700 1 -- 172.21.15.7:0/3694433333 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200cbbf0 con 0x7f6f20156160 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f250aa700 1 --2- 172.21.15.7:0/3694433333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f2006e900 0x7f6f2006c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f1ffff700 1 --2- 172.21.15.7:0/3694433333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f20156160 0x7f6f2006e3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0098e0 tx=0x7f6f0c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7927c62e622eeb server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f1f7fe700 1 -- 172.21.15.7:0/3694433333 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00b640 con 0x7f6f20156160 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f1f7fe700 1 -- 172.21.15.7:0/3694433333 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c00b7a0 con 0x7f6f20156160 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.487+0000 7f6f1f7fe700 1 -- 172.21.15.7:0/3694433333 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00bb40 con 0x7f6f20156160 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.488+0000 7f6f26b0d700 1 -- 172.21.15.7:0/3694433333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f20156160 msgr2=0x7f6f2006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.488+0000 7f6f26b0d700 1 --2- 172.21.15.7:0/3694433333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f20156160 0x7f6f2006e3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0098e0 tx=0x7f6f0c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.488+0000 7f6f26b0d700 1 -- 172.21.15.7:0/3694433333 shutdown_connections 2023-10-07T07:58:41.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.488+0000 7f6f26b0d700 1 --2- 172.21.15.7:0/3694433333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f2006e900 0x7f6f2006c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.488+0000 7f6f26b0d700 1 --2- 172.21.15.7:0/3694433333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f20156160 0x7f6f2006e3c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.488+0000 7f6f26b0d700 1 -- 172.21.15.7:0/3694433333 >> 172.21.15.7:0/3694433333 conn(0x7f6f20068100 msgr2=0x7f6f20068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:41.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.488+0000 7f6f26b0d700 1 -- 172.21.15.7:0/3694433333 shutdown_connections 2023-10-07T07:58:41.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.488+0000 7f6f26b0d700 1 -- 172.21.15.7:0/3694433333 wait complete. 2023-10-07T07:58:41.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.489+0000 7f6f26b0d700 1 Processor -- start 2023-10-07T07:58:41.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.489+0000 7f6f26b0d700 1 -- start start 2023-10-07T07:58:41.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f26b0d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f2006e900 0x7f6f200ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f26b0d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f20155d90 0x7f6f200aa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f26b0d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f20156160 0x7f6f2016a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f26b0d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200ace50 con 0x7f6f2006e900 2023-10-07T07:58:41.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f26b0d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200acc40 con 0x7f6f20155d90 2023-10-07T07:58:41.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f26b0d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200ae300 con 0x7f6f20156160 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f248a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f2006e900 0x7f6f200ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f248a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f2006e900 0x7f6f200ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56302/0 (socket says 172.21.15.7:56302) 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f248a9700 1 -- 172.21.15.7:0/1235781596 learned_addr learned my addr 172.21.15.7:0/1235781596 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.490+0000 7f6f1ffff700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f20155d90 0x7f6f200aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f250aa700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f20156160 0x7f6f2016a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f248a9700 1 -- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f20156160 msgr2=0x7f6f2016a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f248a9700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f20156160 0x7f6f2016a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f248a9700 1 -- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f20155d90 msgr2=0x7f6f200aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f248a9700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f20155d90 0x7f6f200aa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f248a9700 1 -- 172.21.15.7:0/1235781596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f18009a20 con 0x7f6f2006e900 2023-10-07T07:58:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f248a9700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f2006e900 0x7f6f200ac700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400c5b0 tx=0x7f6f14001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f1cff9700 1 -- 172.21.15.7:0/1235781596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14005b30 con 0x7f6f2006e900 2023-10-07T07:58:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f1cff9700 1 -- 172.21.15.7:0/1235781596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f14005c90 con 0x7f6f2006e900 2023-10-07T07:58:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c009420 con 0x7f6f2006e900 2023-10-07T07:58:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f1cff9700 1 -- 172.21.15.7:0/1235781596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14007300 con 0x7f6f2006e900 2023-10-07T07:58:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.491+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f200ac310 con 0x7f6f2006e900 2023-10-07T07:58:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.492+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2004c3c0 con 0x7f6f2006e900 2023-10-07T07:58:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.493+0000 7f6f1cff9700 1 -- 172.21.15.7:0/1235781596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f1401b3b0 con 0x7f6f2006e900 2023-10-07T07:58:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.493+0000 7f6f1cff9700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f04066f30 0x7f6f040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.493+0000 7f6f1cff9700 1 -- 172.21.15.7:0/1235781596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f140839c0 con 0x7f6f2006e900 2023-10-07T07:58:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.493+0000 7f6f1ffff700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f04066f30 0x7f6f040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.494+0000 7f6f1ffff700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f04066f30 0x7f6f040693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0093f0 tx=0x7f6f0c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.498+0000 7f6f1cff9700 1 -- 172.21.15.7:0/1235781596 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f14050c50 con 0x7f6f2006e900 2023-10-07T07:58:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:41.750+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6f20150460 con 0x7f6f2006e900 2023-10-07T07:58:42.555 INFO:tasks.ceph.mds.a.smithi007.stderr:2023-10-07T07:58:42.554+0000 7f6889de9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T07:58:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.590+0000 7f6f1cff9700 1 -- 172.21.15.7:0/1235781596 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6f14017020 con 0x7f6f2006e900 2023-10-07T07:58:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f04066f30 msgr2=0x7f6f040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f04066f30 0x7f6f040693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0093f0 tx=0x7f6f0c017000 comp rx=0 tx=0).stop 2023-10-07T07:58:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f2006e900 msgr2=0x7f6f200ac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f2006e900 0x7f6f200ac700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400c5b0 tx=0x7f6f14001070 comp rx=0 tx=0).stop 2023-10-07T07:58:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 shutdown_connections 2023-10-07T07:58:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f04066f30 0x7f6f040693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f20156160 0x7f6f2016a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f2006e900 0x7f6f200ac700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 --2- 172.21.15.7:0/1235781596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f20155d90 0x7f6f200aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.593+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 >> 172.21.15.7:0/1235781596 conn(0x7f6f20068100 msgr2=0x7f6f20150d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.594+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 shutdown_connections 2023-10-07T07:58:42.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.594+0000 7f6f26b0d700 1 -- 172.21.15.7:0/1235781596 wait complete. 2023-10-07T07:58:42.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-07T07:58:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.747+0000 7f0adb31f700 1 Processor -- start 2023-10-07T07:58:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.748+0000 7f0adb31f700 1 -- start start 2023-10-07T07:58:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0adb31f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ad4156f30 0x7f0ad40b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0adb31f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad40b22a0 0x7f0ad40afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0adb31f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ad4157300 0x7f0ad40affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0adb31f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad40b18d0 con 0x7f0ad4156f30 2023-10-07T07:58:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0adb31f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad40b0fa0 con 0x7f0ad4157300 2023-10-07T07:58:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0adb31f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad40b1280 con 0x7f0ad40b22a0 2023-10-07T07:58:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad90bb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ad4156f30 0x7f0ad40b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad90bb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ad4156f30 0x7f0ad40b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56310/0 (socket says 172.21.15.7:56310) 2023-10-07T07:58:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad90bb700 1 -- 172.21.15.7:0/524606376 learned_addr learned my addr 172.21.15.7:0/524606376 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad98bc700 1 --2- 172.21.15.7:0/524606376 >> v2:172.21.15.172:3300/0 conn(0x7f0ad4157300 0x7f0ad40affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad88ba700 1 --2- 172.21.15.7:0/524606376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad40b22a0 0x7f0ad40afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad90bb700 1 -- 172.21.15.7:0/524606376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad40b22a0 msgr2=0x7f0ad40afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad90bb700 1 --2- 172.21.15.7:0/524606376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad40b22a0 0x7f0ad40afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad90bb700 1 -- 172.21.15.7:0/524606376 >> v2:172.21.15.172:3300/0 conn(0x7f0ad4157300 msgr2=0x7f0ad40affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad90bb700 1 --2- 172.21.15.7:0/524606376 >> v2:172.21.15.172:3300/0 conn(0x7f0ad4157300 0x7f0ad40affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad90bb700 1 -- 172.21.15.7:0/524606376 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad40b1560 con 0x7f0ad4156f30 2023-10-07T07:58:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad98bc700 1 --2- 172.21.15.7:0/524606376 >> v2:172.21.15.172:3300/0 conn(0x7f0ad4157300 0x7f0ad40affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad88ba700 1 --2- 172.21.15.7:0/524606376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad40b22a0 0x7f0ad40afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0ad90bb700 1 --2- 172.21.15.7:0/524606376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad4156f30 0x7f0ad40b1d60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ad406f090 tx=0x7f0ad0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18eb035376cc13a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.749+0000 7f0acbfff700 1 -- 172.21.15.7:0/524606376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000b860 con 0x7f0ad4156f30 2023-10-07T07:58:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0acbfff700 1 -- 172.21.15.7:0/524606376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad000b9c0 con 0x7f0ad4156f30 2023-10-07T07:58:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0acbfff700 1 -- 172.21.15.7:0/524606376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000bd80 con 0x7f0ad4156f30 2023-10-07T07:58:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0adb31f700 1 -- 172.21.15.7:0/524606376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad4156f30 msgr2=0x7f0ad40b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0adb31f700 1 --2- 172.21.15.7:0/524606376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad4156f30 0x7f0ad40b1d60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ad406f090 tx=0x7f0ad0009250 comp rx=0 tx=0).stop 2023-10-07T07:58:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0adb31f700 1 -- 172.21.15.7:0/524606376 shutdown_connections 2023-10-07T07:58:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0adb31f700 1 --2- 172.21.15.7:0/524606376 >> v2:172.21.15.172:3300/0 conn(0x7f0ad4157300 0x7f0ad40affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0adb31f700 1 --2- 172.21.15.7:0/524606376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad4156f30 0x7f0ad40b1d60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0adb31f700 1 --2- 172.21.15.7:0/524606376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad40b22a0 0x7f0ad40afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0adb31f700 1 -- 172.21.15.7:0/524606376 >> 172.21.15.7:0/524606376 conn(0x7f0ad406d350 msgr2=0x7f0ad406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0adb31f700 1 -- 172.21.15.7:0/524606376 shutdown_connections 2023-10-07T07:58:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.750+0000 7f0adb31f700 1 -- 172.21.15.7:0/524606376 wait complete. 2023-10-07T07:58:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.752+0000 7f0adb31f700 1 Processor -- start 2023-10-07T07:58:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.752+0000 7f0adb31f700 1 -- start start 2023-10-07T07:58:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.753+0000 7f0adb31f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad40b22a0 0x7f0ad41561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.753+0000 7f0adb31f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad4156f30 0x7f0ad414d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.753+0000 7f0adb31f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad4157300 0x7f0ad41529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.753+0000 7f0adb31f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad4152f20 con 0x7f0ad40b22a0 2023-10-07T07:58:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.753+0000 7f0adb31f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad4154f20 con 0x7f0ad4156f30 2023-10-07T07:58:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.753+0000 7f0adb31f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad4155200 con 0x7f0ad4157300 2023-10-07T07:58:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.753+0000 7f0ad90bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad40b22a0 0x7f0ad41561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.753+0000 7f0ad90bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad40b22a0 0x7f0ad41561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56316/0 (socket says 172.21.15.7:56316) 2023-10-07T07:58:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.753+0000 7f0ad90bb700 1 -- 172.21.15.7:0/2543853814 learned_addr learned my addr 172.21.15.7:0/2543853814 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ad90bb700 1 -- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad4157300 msgr2=0x7f0ad41529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ad90bb700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad4157300 0x7f0ad41529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ad88ba700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad4156f30 0x7f0ad414d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ad90bb700 1 -- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad4156f30 msgr2=0x7f0ad414d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ad90bb700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad4156f30 0x7f0ad414d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ad90bb700 1 -- 172.21.15.7:0/2543853814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00094f0 con 0x7f0ad40b22a0 2023-10-07T07:58:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ad88ba700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad4156f30 0x7f0ad414d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ad90bb700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad40b22a0 0x7f0ad41561f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b3c0 tx=0x7f0ad0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ac97fa700 1 -- 172.21.15.7:0/2543853814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad001f070 con 0x7f0ad40b22a0 2023-10-07T07:58:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ac97fa700 1 -- 172.21.15.7:0/2543853814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad001a420 con 0x7f0ad40b22a0 2023-10-07T07:58:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad4152100 con 0x7f0ad40b22a0 2023-10-07T07:58:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0ac97fa700 1 -- 172.21.15.7:0/2543853814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad001a7e0 con 0x7f0ad40b22a0 2023-10-07T07:58:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.754+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad4155690 con 0x7f0ad40b22a0 2023-10-07T07:58:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.755+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad404c3c0 con 0x7f0ad40b22a0 2023-10-07T07:58:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.755+0000 7f0ac97fa700 1 -- 172.21.15.7:0/2543853814 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ad001ab90 con 0x7f0ad40b22a0 2023-10-07T07:58:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.756+0000 7f0ac97fa700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0ab8066f30 0x7f0ab80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.756+0000 7f0ac97fa700 1 -- 172.21.15.7:0/2543853814 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0ad0085b00 con 0x7f0ad40b22a0 2023-10-07T07:58:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.756+0000 7f0ad88ba700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0ab8066f30 0x7f0ab80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.757+0000 7f0ad88ba700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0ab8066f30 0x7f0ab80693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ad416ac50 tx=0x7f0ac4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:42.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:42.760+0000 7f0ac97fa700 1 -- 172.21.15.7:0/2543853814 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ad0052df0 con 0x7f0ad40b22a0 2023-10-07T07:58:43.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.009+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0ad40b3d50 con 0x7f0ad40b22a0 2023-10-07T07:58:43.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.592+0000 7f0ac97fa700 1 -- 172.21.15.7:0/2543853814 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0ad00150c0 con 0x7f0ad40b22a0 2023-10-07T07:58:43.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.595+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0ab8066f30 msgr2=0x7f0ab80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:43.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.595+0000 7f0adb31f700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0ab8066f30 0x7f0ab80693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ad416ac50 tx=0x7f0ac4008040 comp rx=0 tx=0).stop 2023-10-07T07:58:43.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.595+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad40b22a0 msgr2=0x7f0ad41561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:43.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.595+0000 7f0adb31f700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad40b22a0 0x7f0ad41561f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b3c0 tx=0x7f0ad0017000 comp rx=0 tx=0).stop 2023-10-07T07:58:43.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.596+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 shutdown_connections 2023-10-07T07:58:43.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.596+0000 7f0adb31f700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0ab8066f30 0x7f0ab80693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.596+0000 7f0adb31f700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad4157300 0x7f0ad41529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.596+0000 7f0adb31f700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad40b22a0 0x7f0ad41561f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.596+0000 7f0adb31f700 1 --2- 172.21.15.7:0/2543853814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad4156f30 0x7f0ad414d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.596+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 >> 172.21.15.7:0/2543853814 conn(0x7f0ad406d350 msgr2=0x7f0ad40ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:43.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.596+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 shutdown_connections 2023-10-07T07:58:43.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.596+0000 7f0adb31f700 1 -- 172.21.15.7:0/2543853814 wait complete. 2023-10-07T07:58:43.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:58:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.748+0000 7fa8740d9700 1 Processor -- start 2023-10-07T07:58:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.748+0000 7fa8740d9700 1 -- start start 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.749+0000 7fa8740d9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c159380 0x7fa86c0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.749+0000 7fa8740d9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0b4820 con 0x7fa86c0b4270 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.749+0000 7fa8740d9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0b5000 con 0x7fa86c0d0250 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.749+0000 7fa8740d9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0b52e0 con 0x7fa86c159380 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.749+0000 7fa872676700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa86c0b4270 0x7fa86c0ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58132/0 (socket says 172.21.15.7:58132) 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.749+0000 7fa872676700 1 -- 172.21.15.7:0/974604362 learned_addr learned my addr 172.21.15.7:0/974604362 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa871e75700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c159380 0x7fa86c0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884276924 0 0) 0x7fa86c0b4820 con 0x7fa86c0b4270 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa858003570 con 0x7fa86c0b4270 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037672005 0 0) 0x7fa86c0b5000 con 0x7fa86c0d0250 2023-10-07T07:58:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa86c0b4820 con 0x7fa86c0d0250 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2452307500 0 0) 0x7fa858003570 con 0x7fa86c0b4270 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c159380 msgr2=0x7fa86c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c159380 0x7fa86c0b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 >> v1:172.21.15.172:6789/0 conn(0x7fa86c0d0250 legacy=0x7fa86c0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0b55c0 con 0x7fa86c0b4270 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7fa8640017f0 con 0x7fa86c0b4270 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 >> v1:172.21.15.7:6789/0 conn(0x7fa86c0b4270 legacy=0x7fa86c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c159380 0x7fa858003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8580041d0 0x7fa858004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa858004d50 0x7fa8580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0b52e0 con 0x7fa858004d50 2023-10-07T07:58:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85800bbf0 con 0x7fa8580041d0 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85800bdb0 con 0x7fa86c159380 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa872676700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa858004d50 0x7fa8580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa871e75700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c159380 0x7fa858003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.750+0000 7fa871674700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8580041d0 0x7fa858004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa872676700 1 -- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c159380 msgr2=0x7fa858003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa872676700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c159380 0x7fa858003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa872676700 1 -- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8580041d0 msgr2=0x7fa858004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa872676700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8580041d0 0x7fa858004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa872676700 1 -- 172.21.15.7:0/974604362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa858003570 con 0x7fa858004d50 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa872676700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa858004d50 0x7fa8580051c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa86400a800 tx=0x7fa8640020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1351a011da81d0ac server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640071a0 con 0x7fa858004d50 2023-10-07T07:58:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa864007300 con 0x7fa858004d50 2023-10-07T07:58:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa870e73700 1 -- 172.21.15.7:0/974604362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640076c0 con 0x7fa858004d50 2023-10-07T07:58:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa8740d9700 1 -- 172.21.15.7:0/974604362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa858004d50 msgr2=0x7fa8580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa8740d9700 1 --2- 172.21.15.7:0/974604362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa858004d50 0x7fa8580051c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa86400a800 tx=0x7fa8640020c0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa872676700 1 -- 172.21.15.7:0/974604362 reap_dead start 2023-10-07T07:58:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa8740d9700 1 -- 172.21.15.7:0/974604362 shutdown_connections 2023-10-07T07:58:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.751+0000 7fa8740d9700 1 -- 172.21.15.7:0/974604362 >> 172.21.15.7:0/974604362 conn(0x7fa86c067be0 msgr2=0x7fa86c067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.752+0000 7fa8740d9700 1 -- 172.21.15.7:0/974604362 shutdown_connections 2023-10-07T07:58:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.752+0000 7fa8740d9700 1 -- 172.21.15.7:0/974604362 wait complete. 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.753+0000 7fa8740d9700 1 Processor -- start 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.753+0000 7fa8740d9700 1 -- start start 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa8740d9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa858004d50 0x7fa86c150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa8740d9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c15ecd0 0x7fa86c15f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa8740d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86c155440 0x7fa86c1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa8740d9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c15f680 con 0x7fa86c155440 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa8740d9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c156190 con 0x7fa858004d50 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa8740d9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c156470 con 0x7fa86c15ecd0 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa872676700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86c155440 0x7fa86c1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa872676700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86c155440 0x7fa86c1558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56328/0 (socket says 172.21.15.7:56328) 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa872676700 1 -- 172.21.15.7:0/1395942235 learned_addr learned my addr 172.21.15.7:0/1395942235 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa871e75700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa858004d50 0x7fa86c150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa871674700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c15ecd0 0x7fa86c15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa872676700 1 -- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c15ecd0 msgr2=0x7fa86c15f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa872676700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c15ecd0 0x7fa86c15f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa872676700 1 -- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa858004d50 msgr2=0x7fa86c150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa872676700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa858004d50 0x7fa86c150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa872676700 1 -- 172.21.15.7:0/1395942235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c156750 con 0x7fa86c155440 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa872676700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86c155440 0x7fa86c1558b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f80 tx=0x7fa864016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.754+0000 7fa861ffb700 1 -- 172.21.15.7:0/1395942235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640011c0 con 0x7fa86c155440 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.755+0000 7fa861ffb700 1 -- 172.21.15.7:0/1395942235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa864001340 con 0x7fa86c155440 2023-10-07T07:58:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.755+0000 7fa861ffb700 1 -- 172.21.15.7:0/1395942235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864020640 con 0x7fa86c155440 2023-10-07T07:58:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.755+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c156a30 con 0x7fa86c155440 2023-10-07T07:58:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.755+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c151300 con 0x7fa86c155440 2023-10-07T07:58:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.756+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c156f50 con 0x7fa86c155440 2023-10-07T07:58:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.756+0000 7fa861ffb700 1 -- 172.21.15.7:0/1395942235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8640209f0 con 0x7fa86c155440 2023-10-07T07:58:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.757+0000 7fa861ffb700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa850066f30 0x7fa8500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.757+0000 7fa861ffb700 1 -- 172.21.15.7:0/1395942235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa864085e90 con 0x7fa86c155440 2023-10-07T07:58:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.757+0000 7fa871e75700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa850066f30 0x7fa8500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.757+0000 7fa871e75700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa850066f30 0x7fa8500693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa868006f00 tx=0x7fa8680091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:43.761+0000 7fa861ffb700 1 -- 172.21.15.7:0/1395942235 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa86c156f50 con 0x7fa86c155440 2023-10-07T07:58:44.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.012+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa86c156f50 con 0x7fa86c155440 2023-10-07T07:58:44.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.013+0000 7fa861ffb700 1 -- 172.21.15.7:0/1395942235 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fa86c156f50 con 0x7fa86c155440 2023-10-07T07:58:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:44.014 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/580346028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":580346028},{"type":"v1","addr":"172.21.15.172:6837","nonce":580346028}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3012863722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3012863722},{"type":"v1","addr":"172.21.15.7:6835","nonce":3012863722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4947,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3733075096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3733075096},{"type":"v1","addr":"172.21.15.172:6836","nonce":3733075096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:58:40.003670+0000","modified":"2023-10-07T07:58:43.589885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4959":{"gid":4959,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/786560894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":786560894},{"type":"v1","addr":"172.21.15.7:6834","nonce":786560894}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T07:58:44.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa850066f30 msgr2=0x7fa8500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa850066f30 0x7fa8500693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa868006f00 tx=0x7fa8680091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86c155440 msgr2=0x7fa86c1558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86c155440 0x7fa86c1558b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f80 tx=0x7fa864016000 comp rx=0 tx=0).stop 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 shutdown_connections 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa850066f30 0x7fa8500693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86c155440 0x7fa86c1558b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa858004d50 0x7fa86c150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 --2- 172.21.15.7:0/1395942235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa86c15ecd0 0x7fa86c15f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 >> 172.21.15.7:0/1395942235 conn(0x7fa86c067be0 msgr2=0x7fa86c154500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 shutdown_connections 2023-10-07T07:58:44.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.015+0000 7fa8740d9700 1 -- 172.21.15.7:0/1395942235 wait complete. 2023-10-07T07:58:44.018 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-10-07T07:58:44.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:58:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.161+0000 7f1b159ea700 1 Processor -- start 2023-10-07T07:58:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.161+0000 7f1b159ea700 1 -- start start 2023-10-07T07:58:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.161+0000 7f1b159ea700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1b10159380 0x7f1b100b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b159ea700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b100b4270 0x7f1b100d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b159ea700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b100d0be0 con 0x7f1b100d0250 2023-10-07T07:58:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b159ea700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b100b4820 con 0x7f1b10159380 2023-10-07T07:58:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b159ea700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100b5000 con 0x7f1b100b4270 2023-10-07T07:58:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0f7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b100b4270 0x7f1b100d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0effd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1b10159380 0x7f1b100b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0e7fc700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1b100d0250 0x7f1b100b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:45544/0 (socket says 172.21.15.7:45544) 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0e7fc700 1 -- 172.21.15.7:0/99321273 learned_addr learned my addr 172.21.15.7:0/99321273 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0f7fe700 1 -- 172.21.15.7:0/99321273 >> v2:172.21.15.172:3300/0 conn(0x7f1b10159380 msgr2=0x7f1b100b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0f7fe700 1 --2- 172.21.15.7:0/99321273 >> v2:172.21.15.172:3300/0 conn(0x7f1b10159380 0x7f1b100b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0f7fe700 1 -- 172.21.15.7:0/99321273 >> v1:172.21.15.172:6789/0 conn(0x7f1b100d0250 legacy=0x7f1b100b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0f7fe700 1 -- 172.21.15.7:0/99321273 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b100b52e0 con 0x7f1b100b4270 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0dffb700 1 -- 172.21.15.7:0/99321273 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989176816 0 0) 0x7f1b100d0be0 con 0x7f1b100d0250 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.162+0000 7f1b0effd700 1 --2- 172.21.15.7:0/99321273 >> v2:172.21.15.172:3300/0 conn(0x7f1b10159380 0x7f1b100b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b0f7fe700 1 --2- 172.21.15.7:0/99321273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b100b4270 0x7f1b100d06a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009e60 tx=0x7f1b08009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b98ac19e2332ded4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b0dffb700 1 -- 172.21.15.7:0/99321273 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800e070 con 0x7f1b100b4270 2023-10-07T07:58:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b0dffb700 1 -- 172.21.15.7:0/99321273 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0800bb60 con 0x7f1b100b4270 2023-10-07T07:58:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b0dffb700 1 -- 172.21.15.7:0/99321273 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08012430 con 0x7f1b100b4270 2023-10-07T07:58:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b159ea700 1 -- 172.21.15.7:0/99321273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b100b4270 msgr2=0x7f1b100d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b159ea700 1 --2- 172.21.15.7:0/99321273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b100b4270 0x7f1b100d06a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009e60 tx=0x7f1b08009780 comp rx=0 tx=0).stop 2023-10-07T07:58:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b159ea700 1 -- 172.21.15.7:0/99321273 shutdown_connections 2023-10-07T07:58:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b159ea700 1 --2- 172.21.15.7:0/99321273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b100b4270 0x7f1b100d06a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b159ea700 1 --2- 172.21.15.7:0/99321273 >> v2:172.21.15.172:3300/0 conn(0x7f1b10159380 0x7f1b100b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.163+0000 7f1b159ea700 1 -- 172.21.15.7:0/99321273 >> 172.21.15.7:0/99321273 conn(0x7f1b10067be0 msgr2=0x7f1b10067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.164+0000 7f1b159ea700 1 -- 172.21.15.7:0/99321273 shutdown_connections 2023-10-07T07:58:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.164+0000 7f1b159ea700 1 -- 172.21.15.7:0/99321273 wait complete. 2023-10-07T07:58:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.165+0000 7f1b159ea700 1 Processor -- start 2023-10-07T07:58:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.165+0000 7f1b159ea700 1 -- start start 2023-10-07T07:58:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.166+0000 7f1b159ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b100b4270 0x7f1b1016c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.166+0000 7f1b159ea700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b100d0250 0x7f1b1016ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.166+0000 7f1b159ea700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b10159380 0x7f1b100cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.166+0000 7f1b159ea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1015c0b0 con 0x7f1b100b4270 2023-10-07T07:58:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.166+0000 7f1b159ea700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1015e890 con 0x7f1b100d0250 2023-10-07T07:58:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.166+0000 7f1b159ea700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1015bdd0 con 0x7f1b10159380 2023-10-07T07:58:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b100b4270 0x7f1b1016c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b100b4270 0x7f1b1016c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56330/0 (socket says 172.21.15.7:56330) 2023-10-07T07:58:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0effd700 1 -- 172.21.15.7:0/1424032197 learned_addr learned my addr 172.21.15.7:0/1424032197 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0e7fc700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b100d0250 0x7f1b1016ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0f7fe700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b10159380 0x7f1b100cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0effd700 1 -- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b10159380 msgr2=0x7f1b100cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0effd700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b10159380 0x7f1b100cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0effd700 1 -- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b100d0250 msgr2=0x7f1b1016ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0effd700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b100d0250 0x7f1b1016ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0effd700 1 -- 172.21.15.7:0/1424032197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b00009420 con 0x7f1b100b4270 2023-10-07T07:58:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b0effd700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b100b4270 0x7f1b1016c790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000ec80 tx=0x7f1b000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1af6ffd700 1 -- 172.21.15.7:0/1424032197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000c850 con 0x7f1b100b4270 2023-10-07T07:58:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1af6ffd700 1 -- 172.21.15.7:0/1424032197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0000c9b0 con 0x7f1b100b4270 2023-10-07T07:58:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1af6ffd700 1 -- 172.21.15.7:0/1424032197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000cd30 con 0x7f1b100b4270 2023-10-07T07:58:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.167+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b08009a20 con 0x7f1b100b4270 2023-10-07T07:58:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.168+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1015e520 con 0x7f1b100b4270 2023-10-07T07:58:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.168+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1015eb70 con 0x7f1b100b4270 2023-10-07T07:58:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.169+0000 7f1af6ffd700 1 -- 172.21.15.7:0/1424032197 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b00013680 con 0x7f1b100b4270 2023-10-07T07:58:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.169+0000 7f1af6ffd700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1afc066f30 0x7f1afc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.169+0000 7f1af6ffd700 1 -- 172.21.15.7:0/1424032197 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1b00084770 con 0x7f1b100b4270 2023-10-07T07:58:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.170+0000 7f1b0e7fc700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1afc066f30 0x7f1afc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.170+0000 7f1b0e7fc700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1afc066f30 0x7f1afc0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b1015dfd0 tx=0x7f1b04007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:44.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.174+0000 7f1af6ffd700 1 -- 172.21.15.7:0/1424032197 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b00055470 con 0x7f1b100b4270 2023-10-07T07:58:44.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.381+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b100716c0 con 0x7f1b100b4270 2023-10-07T07:58:44.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.382+0000 7f1af6ffd700 1 -- 172.21.15.7:0/1424032197 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f1b00055000 con 0x7f1b100b4270 2023-10-07T07:58:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:44.383 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:58:41.578137+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:58:37.928486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T07:58:38.977756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:58:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1afc066f30 msgr2=0x7f1afc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1afc066f30 0x7f1afc0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b1015dfd0 tx=0x7f1b04007d30 comp rx=0 tx=0).stop 2023-10-07T07:58:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b100b4270 msgr2=0x7f1b1016c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b100b4270 0x7f1b1016c790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000ec80 tx=0x7f1b000091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 shutdown_connections 2023-10-07T07:58:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1afc066f30 0x7f1afc0693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b10159380 0x7f1b100cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b100b4270 0x7f1b1016c790 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 --2- 172.21.15.7:0/1424032197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b100d0250 0x7f1b1016ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.384+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 >> 172.21.15.7:0/1424032197 conn(0x7f1b10067be0 msgr2=0x7f1b100ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.385+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 shutdown_connections 2023-10-07T07:58:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.385+0000 7f1b159ea700 1 -- 172.21.15.7:0/1424032197 wait complete. 2023-10-07T07:58:44.400 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-07T07:58:44.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T07:58:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.531+0000 7f62345a0700 1 Processor -- start 2023-10-07T07:58:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.531+0000 7f62345a0700 1 -- start start 2023-10-07T07:58:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f62345a0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f622c159310 0x7f622c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f62345a0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c1596e0 0x7f622c06b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f62345a0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622c068100 con 0x7f622c0ced30 2023-10-07T07:58:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f62345a0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c068350 con 0x7f622c159310 2023-10-07T07:58:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f62345a0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c068630 con 0x7f622c1596e0 2023-10-07T07:58:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f623233c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f622c159310 0x7f622c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f6231b3b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f622c0ced30 0x7f622c0c3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58140/0 (socket says 172.21.15.7:58140) 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f623233c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f622c159310 0x7f622c0c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56342/0 (socket says 172.21.15.7:56342) 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f6231b3b700 1 -- 172.21.15.7:0/3767018872 learned_addr learned my addr 172.21.15.7:0/3767018872 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f6232b3d700 1 --2- 172.21.15.7:0/3767018872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c1596e0 0x7f622c06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f623233c700 1 -- 172.21.15.7:0/3767018872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c1596e0 msgr2=0x7f622c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f623233c700 1 --2- 172.21.15.7:0/3767018872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c1596e0 0x7f622c06b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f623233c700 1 -- 172.21.15.7:0/3767018872 >> v1:172.21.15.7:6789/0 conn(0x7f622c0ced30 legacy=0x7f622c0c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f623233c700 1 -- 172.21.15.7:0/3767018872 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c068910 con 0x7f622c159310 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f623133a700 1 -- 172.21.15.7:0/3767018872 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098215216 0 0) 0x7f622c068100 con 0x7f622c0ced30 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.532+0000 7f623233c700 1 --2- 172.21.15.7:0/3767018872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 0x7f622c0c86e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f62280098e0 tx=0x7f62280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aef833ed96797ae server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f623133a700 1 -- 172.21.15.7:0/3767018872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6228009e80 con 0x7f622c159310 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f623133a700 1 -- 172.21.15.7:0/3767018872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622800b860 con 0x7f622c159310 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f623133a700 1 -- 172.21.15.7:0/3767018872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800bbe0 con 0x7f622c159310 2023-10-07T07:58:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f62345a0700 1 -- 172.21.15.7:0/3767018872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 msgr2=0x7f622c0c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f62345a0700 1 --2- 172.21.15.7:0/3767018872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 0x7f622c0c86e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f62280098e0 tx=0x7f62280091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f62345a0700 1 -- 172.21.15.7:0/3767018872 shutdown_connections 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f62345a0700 1 --2- 172.21.15.7:0/3767018872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c1596e0 0x7f622c06b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f62345a0700 1 --2- 172.21.15.7:0/3767018872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 0x7f622c0c86e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f62345a0700 1 -- 172.21.15.7:0/3767018872 >> 172.21.15.7:0/3767018872 conn(0x7f622c067ac0 msgr2=0x7f622c0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f62345a0700 1 -- 172.21.15.7:0/3767018872 shutdown_connections 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.533+0000 7f62345a0700 1 -- 172.21.15.7:0/3767018872 wait complete. 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.534+0000 7f62345a0700 1 Processor -- start 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f62345a0700 1 -- start start 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f62345a0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 0x7f622c0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f62345a0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c1596e0 0x7f622c1566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f62345a0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c156c30 0x7f622c0c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f62345a0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1570d0 con 0x7f622c159310 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f62345a0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1573b0 con 0x7f622c1596e0 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f62345a0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c157690 con 0x7f622c156c30 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f623233c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 0x7f622c0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f623233c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 0x7f622c0c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56354/0 (socket says 172.21.15.7:56354) 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.535+0000 7f623233c700 1 -- 172.21.15.7:0/2487324652 learned_addr learned my addr 172.21.15.7:0/2487324652 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f6231b3b700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c1596e0 0x7f622c1566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f6232b3d700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c156c30 0x7f622c0c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f623233c700 1 -- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c156c30 msgr2=0x7f622c0c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f623233c700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c156c30 0x7f622c0c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f623233c700 1 -- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c1596e0 msgr2=0x7f622c1566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f623233c700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c1596e0 0x7f622c1566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f623233c700 1 -- 172.21.15.7:0/2487324652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228009420 con 0x7f622c159310 2023-10-07T07:58:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f6231b3b700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c1596e0 0x7f622c1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f623233c700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 0x7f622c0c7e30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f622c0cbc20 tx=0x7f6228017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f62227fc700 1 -- 172.21.15.7:0/2487324652 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800bbe0 con 0x7f622c159310 2023-10-07T07:58:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f62227fc700 1 -- 172.21.15.7:0/2487324652 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6228011420 con 0x7f622c159310 2023-10-07T07:58:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c157970 con 0x7f622c159310 2023-10-07T07:58:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f62227fc700 1 -- 172.21.15.7:0/2487324652 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62280117e0 con 0x7f622c159310 2023-10-07T07:58:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.536+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c0c8370 con 0x7f622c159310 2023-10-07T07:58:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.537+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c157e90 con 0x7f622c159310 2023-10-07T07:58:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.538+0000 7f62227fc700 1 -- 172.21.15.7:0/2487324652 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6228011bd0 con 0x7f622c159310 2023-10-07T07:58:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.538+0000 7f62227fc700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6210066f60 0x7f6210069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.538+0000 7f62227fc700 1 -- 172.21.15.7:0/2487324652 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6228085f70 con 0x7f622c159310 2023-10-07T07:58:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.538+0000 7f6231b3b700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6210066f60 0x7f6210069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.539+0000 7f6231b3b700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6210066f60 0x7f6210069410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f622c0ce0a0 tx=0x7f621c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.543+0000 7f62227fc700 1 -- 172.21.15.7:0/2487324652 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62280b75a0 con 0x7f622c159310 2023-10-07T07:58:44.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.741+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f622c157520 con 0x7f6210066f60 2023-10-07T07:58:44.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.750+0000 7f62227fc700 1 -- 172.21.15.7:0/2487324652 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271074 (secure 0 0 0) 0x7f622c157520 con 0x7f6210066f60 2023-10-07T07:58:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:44.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6210066f60 msgr2=0x7f6210069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6210066f60 0x7f6210069410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f622c0ce0a0 tx=0x7f621c009040 comp rx=0 tx=0).stop 2023-10-07T07:58:44.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 msgr2=0x7f622c0c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 0x7f622c0c7e30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f622c0cbc20 tx=0x7f6228017000 comp rx=0 tx=0).stop 2023-10-07T07:58:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 shutdown_connections 2023-10-07T07:58:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6210066f60 0x7f6210069410 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c156c30 0x7f622c0c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f622c159310 0x7f622c0c7e30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 --2- 172.21.15.7:0/2487324652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c1596e0 0x7f622c1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 >> 172.21.15.7:0/2487324652 conn(0x7f622c067ac0 msgr2=0x7f622c154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 shutdown_connections 2023-10-07T07:58:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.753+0000 7f62345a0700 1 -- 172.21.15.7:0/2487324652 wait complete. 2023-10-07T07:58:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T07:58:44.768 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2023-10-07T07:58:44.037487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382416,"kb_used_data":4112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592304,"statfs":{"total":773094113280,"available":770654519296,"internally_reserved":0,"allocated":4210688,"data_stored":2322244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.474125"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609287+0000","last_change":"2023-10-07T07:58:40.606566+0000","last_active":"2023-10-07T07:58:40.609287+0000","last_peered":"2023-10-07T07:58:40.609287+0000","last_clean":"2023-10-07T07:58:40.609287+0000","last_became_active":"2023-10-07T07:58:40.606336+0000","last_became_peered":"2023-10-07T07:58:40.606336+0000","last_unstale":"2023-10-07T07:58:40.609287+0000","last_undegraded":"2023-10-07T07:58:40.609287+0000","last_fullsized":"2023-10-07T07:58:40.609287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:31:14.816056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.597711+0000","last_change":"2023-10-07T07:58:40.593477+0000","last_active":"2023-10-07T07:58:40.597711+0000","last_peered":"2023-10-07T07:58:40.597711+0000","last_clean":"2023-10-07T07:58:40.597711+0000","last_became_active":"2023-10-07T07:58:40.593184+0000","last_became_peered":"2023-10-07T07:58:40.593184+0000","last_unstale":"2023-10-07T07:58:40.597711+0000","last_undegraded":"2023-10-07T07:58:40.597711+0000","last_fullsized":"2023-10-07T07:58:40.597711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:01.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600124+0000","last_change":"2023-10-07T07:58:40.596336+0000","last_active":"2023-10-07T07:58:40.600124+0000","last_peered":"2023-10-07T07:58:40.600124+0000","last_clean":"2023-10-07T07:58:40.600124+0000","last_became_active":"2023-10-07T07:58:40.595994+0000","last_became_peered":"2023-10-07T07:58:40.595994+0000","last_unstale":"2023-10-07T07:58:40.600124+0000","last_undegraded":"2023-10-07T07:58:40.600124+0000","last_fullsized":"2023-10-07T07:58:40.600124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:39:28.581635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595449+0000","last_change":"2023-10-07T07:58:40.592783+0000","last_active":"2023-10-07T07:58:40.595449+0000","last_peered":"2023-10-07T07:58:40.595449+0000","last_clean":"2023-10-07T07:58:40.595449+0000","last_became_active":"2023-10-07T07:58:40.592512+0000","last_became_peered":"2023-10-07T07:58:40.592512+0000","last_unstale":"2023-10-07T07:58:40.595449+0000","last_undegraded":"2023-10-07T07:58:40.595449+0000","last_fullsized":"2023-10-07T07:58:40.595449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:35:31.024079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.597077+0000","last_change":"2023-10-07T07:58:40.594194+0000","last_active":"2023-10-07T07:58:40.597077+0000","last_peered":"2023-10-07T07:58:40.597077+0000","last_clean":"2023-10-07T07:58:40.597077+0000","last_became_active":"2023-10-07T07:58:40.593929+0000","last_became_peered":"2023-10-07T07:58:40.593929+0000","last_unstale":"2023-10-07T07:58:40.597077+0000","last_undegraded":"2023-10-07T07:58:40.597077+0000","last_fullsized":"2023-10-07T07:58:40.597077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:05:32.719608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600026+0000","last_change":"2023-10-07T07:58:40.592305+0000","last_active":"2023-10-07T07:58:40.600026+0000","last_peered":"2023-10-07T07:58:40.600026+0000","last_clean":"2023-10-07T07:58:40.600026+0000","last_became_active":"2023-10-07T07:58:40.591952+0000","last_became_peered":"2023-10-07T07:58:40.591952+0000","last_unstale":"2023-10-07T07:58:40.600026+0000","last_undegraded":"2023-10-07T07:58:40.600026+0000","last_fullsized":"2023-10-07T07:58:40.600026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:39:28.581635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609304+0000","last_change":"2023-10-07T07:58:40.605681+0000","last_active":"2023-10-07T07:58:40.609304+0000","last_peered":"2023-10-07T07:58:40.609304+0000","last_clean":"2023-10-07T07:58:40.609304+0000","last_became_active":"2023-10-07T07:58:40.605394+0000","last_became_peered":"2023-10-07T07:58:40.605394+0000","last_unstale":"2023-10-07T07:58:40.609304+0000","last_undegraded":"2023-10-07T07:58:40.609304+0000","last_fullsized":"2023-10-07T07:58:40.609304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:30:21.307510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.607319+0000","last_change":"2023-10-07T07:58:40.594584+0000","last_active":"2023-10-07T07:58:40.607319+0000","last_peered":"2023-10-07T07:58:40.607319+0000","last_clean":"2023-10-07T07:58:40.607319+0000","last_became_active":"2023-10-07T07:58:40.594250+0000","last_became_peered":"2023-10-07T07:58:40.594250+0000","last_unstale":"2023-10-07T07:58:40.607319+0000","last_undegraded":"2023-10-07T07:58:40.607319+0000","last_fullsized":"2023-10-07T07:58:40.607319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:03.887160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600598+0000","last_change":"2023-10-07T07:58:40.597026+0000","last_active":"2023-10-07T07:58:40.600598+0000","last_peered":"2023-10-07T07:58:40.600598+0000","last_clean":"2023-10-07T07:58:40.600598+0000","last_became_active":"2023-10-07T07:58:40.596659+0000","last_became_peered":"2023-10-07T07:58:40.596659+0000","last_unstale":"2023-10-07T07:58:40.600598+0000","last_undegraded":"2023-10-07T07:58:40.600598+0000","last_fullsized":"2023-10-07T07:58:40.600598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:45.999293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595040+0000","last_change":"2023-10-07T07:58:40.590152+0000","last_active":"2023-10-07T07:58:40.595040+0000","last_peered":"2023-10-07T07:58:40.595040+0000","last_clean":"2023-10-07T07:58:40.595040+0000","last_became_active":"2023-10-07T07:58:40.589246+0000","last_became_peered":"2023-10-07T07:58:40.589246+0000","last_unstale":"2023-10-07T07:58:40.595040+0000","last_undegraded":"2023-10-07T07:58:40.595040+0000","last_fullsized":"2023-10-07T07:58:40.595040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:05:32.719608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609772+0000","last_change":"2023-10-07T07:58:40.589682+0000","last_active":"2023-10-07T07:58:40.609772+0000","last_peered":"2023-10-07T07:58:40.609772+0000","last_clean":"2023-10-07T07:58:40.609772+0000","last_became_active":"2023-10-07T07:58:40.589437+0000","last_became_peered":"2023-10-07T07:58:40.589437+0000","last_unstale":"2023-10-07T07:58:40.609772+0000","last_undegraded":"2023-10-07T07:58:40.609772+0000","last_fullsized":"2023-10-07T07:58:40.609772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:39:56.102863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600394+0000","last_change":"2023-10-07T07:58:40.596580+0000","last_active":"2023-10-07T07:58:40.600394+0000","last_peered":"2023-10-07T07:58:40.600394+0000","last_clean":"2023-10-07T07:58:40.600394+0000","last_became_active":"2023-10-07T07:58:40.596270+0000","last_became_peered":"2023-10-07T07:58:40.596270+0000","last_unstale":"2023-10-07T07:58:40.600394+0000","last_undegraded":"2023-10-07T07:58:40.600394+0000","last_fullsized":"2023-10-07T07:58:40.600394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:58:50.796780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.593166+0000","last_change":"2023-10-07T07:58:40.587912+0000","last_active":"2023-10-07T07:58:40.593166+0000","last_peered":"2023-10-07T07:58:40.593166+0000","last_clean":"2023-10-07T07:58:40.593166+0000","last_became_active":"2023-10-07T07:58:40.586751+0000","last_became_peered":"2023-10-07T07:58:40.586751+0000","last_unstale":"2023-10-07T07:58:40.593166+0000","last_undegraded":"2023-10-07T07:58:40.593166+0000","last_fullsized":"2023-10-07T07:58:40.593166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:32:36.691334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.594593+0000","last_change":"2023-10-07T07:58:40.591131+0000","last_active":"2023-10-07T07:58:40.594593+0000","last_peered":"2023-10-07T07:58:40.594593+0000","last_clean":"2023-10-07T07:58:40.594593+0000","last_became_active":"2023-10-07T07:58:40.588875+0000","last_became_peered":"2023-10-07T07:58:40.588875+0000","last_unstale":"2023-10-07T07:58:40.594593+0000","last_undegraded":"2023-10-07T07:58:40.594593+0000","last_fullsized":"2023-10-07T07:58:40.594593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:01.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.599445+0000","last_change":"2023-10-07T07:58:40.590081+0000","last_active":"2023-10-07T07:58:40.599445+0000","last_peered":"2023-10-07T07:58:40.599445+0000","last_clean":"2023-10-07T07:58:40.599445+0000","last_became_active":"2023-10-07T07:58:40.589665+0000","last_became_peered":"2023-10-07T07:58:40.589665+0000","last_unstale":"2023-10-07T07:58:40.599445+0000","last_undegraded":"2023-10-07T07:58:40.599445+0000","last_fullsized":"2023-10-07T07:58:40.599445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:03.887160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604397+0000","last_change":"2023-10-07T07:58:40.593644+0000","last_active":"2023-10-07T07:58:40.604397+0000","last_peered":"2023-10-07T07:58:40.604397+0000","last_clean":"2023-10-07T07:58:40.604397+0000","last_became_active":"2023-10-07T07:58:40.593328+0000","last_became_peered":"2023-10-07T07:58:40.593328+0000","last_unstale":"2023-10-07T07:58:40.604397+0000","last_undegraded":"2023-10-07T07:58:40.604397+0000","last_fullsized":"2023-10-07T07:58:40.604397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:14.250170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604962+0000","last_change":"2023-10-07T07:58:40.589158+0000","last_active":"2023-10-07T07:58:40.604962+0000","last_peered":"2023-10-07T07:58:40.604962+0000","last_clean":"2023-10-07T07:58:40.604962+0000","last_became_active":"2023-10-07T07:58:40.588866+0000","last_became_peered":"2023-10-07T07:58:40.588866+0000","last_unstale":"2023-10-07T07:58:40.604962+0000","last_undegraded":"2023-10-07T07:58:40.604962+0000","last_fullsized":"2023-10-07T07:58:40.604962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:01:14.530812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595309+0000","last_change":"2023-10-07T07:58:40.590179+0000","last_active":"2023-10-07T07:58:40.595309+0000","last_peered":"2023-10-07T07:58:40.595309+0000","last_clean":"2023-10-07T07:58:40.595309+0000","last_became_active":"2023-10-07T07:58:40.589938+0000","last_became_peered":"2023-10-07T07:58:40.589938+0000","last_unstale":"2023-10-07T07:58:40.595309+0000","last_undegraded":"2023-10-07T07:58:40.595309+0000","last_fullsized":"2023-10-07T07:58:40.595309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:31.416135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602551+0000","last_change":"2023-10-07T07:58:40.598761+0000","last_active":"2023-10-07T07:58:40.602551+0000","last_peered":"2023-10-07T07:58:40.602551+0000","last_clean":"2023-10-07T07:58:40.602551+0000","last_became_active":"2023-10-07T07:58:40.598455+0000","last_became_peered":"2023-10-07T07:58:40.598455+0000","last_unstale":"2023-10-07T07:58:40.602551+0000","last_undegraded":"2023-10-07T07:58:40.602551+0000","last_fullsized":"2023-10-07T07:58:40.602551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:45.999293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602439+0000","last_change":"2023-10-07T07:58:40.598934+0000","last_active":"2023-10-07T07:58:40.602439+0000","last_peered":"2023-10-07T07:58:40.602439+0000","last_clean":"2023-10-07T07:58:40.602439+0000","last_became_active":"2023-10-07T07:58:40.598333+0000","last_became_peered":"2023-10-07T07:58:40.598333+0000","last_unstale":"2023-10-07T07:58:40.602439+0000","last_undegraded":"2023-10-07T07:58:40.602439+0000","last_fullsized":"2023-10-07T07:58:40.602439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:30:21.307510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604266+0000","last_change":"2023-10-07T07:58:40.593825+0000","last_active":"2023-10-07T07:58:40.604266+0000","last_peered":"2023-10-07T07:58:40.604266+0000","last_clean":"2023-10-07T07:58:40.604266+0000","last_became_active":"2023-10-07T07:58:40.591700+0000","last_became_peered":"2023-10-07T07:58:40.591700+0000","last_unstale":"2023-10-07T07:58:40.604266+0000","last_undegraded":"2023-10-07T07:58:40.604266+0000","last_fullsized":"2023-10-07T07:58:40.604266+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:05:32.719608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609172+0000","last_change":"2023-10-07T07:58:40.600187+0000","last_active":"2023-10-07T07:58:40.609172+0000","last_peered":"2023-10-07T07:58:40.609172+0000","last_clean":"2023-10-07T07:58:40.609172+0000","last_became_active":"2023-10-07T07:58:40.599706+0000","last_became_peered":"2023-10-07T07:58:40.599706+0000","last_unstale":"2023-10-07T07:58:40.609172+0000","last_undegraded":"2023-10-07T07:58:40.609172+0000","last_fullsized":"2023-10-07T07:58:40.609172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:14.250170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602930+0000","last_change":"2023-10-07T07:58:40.601412+0000","last_active":"2023-10-07T07:58:40.602930+0000","last_peered":"2023-10-07T07:58:40.602930+0000","last_clean":"2023-10-07T07:58:40.602930+0000","last_became_active":"2023-10-07T07:58:40.601100+0000","last_became_peered":"2023-10-07T07:58:40.601100+0000","last_unstale":"2023-10-07T07:58:40.602930+0000","last_undegraded":"2023-10-07T07:58:40.602930+0000","last_fullsized":"2023-10-07T07:58:40.602930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:49:15.409580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609052+0000","last_change":"2023-10-07T07:58:40.599486+0000","last_active":"2023-10-07T07:58:40.609052+0000","last_peered":"2023-10-07T07:58:40.609052+0000","last_clean":"2023-10-07T07:58:40.609052+0000","last_became_active":"2023-10-07T07:58:40.599227+0000","last_became_peered":"2023-10-07T07:58:40.599227+0000","last_unstale":"2023-10-07T07:58:40.609052+0000","last_undegraded":"2023-10-07T07:58:40.609052+0000","last_fullsized":"2023-10-07T07:58:40.609052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:05:32.719608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.603000+0000","last_change":"2023-10-07T07:58:40.596634+0000","last_active":"2023-10-07T07:58:40.603000+0000","last_peered":"2023-10-07T07:58:40.603000+0000","last_clean":"2023-10-07T07:58:40.603000+0000","last_became_active":"2023-10-07T07:58:40.596013+0000","last_became_peered":"2023-10-07T07:58:40.596013+0000","last_unstale":"2023-10-07T07:58:40.603000+0000","last_undegraded":"2023-10-07T07:58:40.603000+0000","last_fullsized":"2023-10-07T07:58:40.603000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:02:38.932523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604298+0000","last_change":"2023-10-07T07:58:40.601340+0000","last_active":"2023-10-07T07:58:40.604298+0000","last_peered":"2023-10-07T07:58:40.604298+0000","last_clean":"2023-10-07T07:58:40.604298+0000","last_became_active":"2023-10-07T07:58:40.601090+0000","last_became_peered":"2023-10-07T07:58:40.601090+0000","last_unstale":"2023-10-07T07:58:40.604298+0000","last_undegraded":"2023-10-07T07:58:40.604298+0000","last_fullsized":"2023-10-07T07:58:40.604298+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:04:39.135155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.605008+0000","last_change":"2023-10-07T07:58:40.604076+0000","last_active":"2023-10-07T07:58:40.605008+0000","last_peered":"2023-10-07T07:58:40.605008+0000","last_clean":"2023-10-07T07:58:40.605008+0000","last_became_active":"2023-10-07T07:58:40.603798+0000","last_became_peered":"2023-10-07T07:58:40.603798+0000","last_unstale":"2023-10-07T07:58:40.605008+0000","last_undegraded":"2023-10-07T07:58:40.605008+0000","last_fullsized":"2023-10-07T07:58:40.605008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:04:39.135155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.607476+0000","last_change":"2023-10-07T07:58:40.594699+0000","last_active":"2023-10-07T07:58:40.607476+0000","last_peered":"2023-10-07T07:58:40.607476+0000","last_clean":"2023-10-07T07:58:40.607476+0000","last_became_active":"2023-10-07T07:58:40.594300+0000","last_became_peered":"2023-10-07T07:58:40.594300+0000","last_unstale":"2023-10-07T07:58:40.607476+0000","last_undegraded":"2023-10-07T07:58:40.607476+0000","last_fullsized":"2023-10-07T07:58:40.607476+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:01.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604288+0000","last_change":"2023-10-07T07:58:40.596980+0000","last_active":"2023-10-07T07:58:40.604288+0000","last_peered":"2023-10-07T07:58:40.604288+0000","last_clean":"2023-10-07T07:58:40.604288+0000","last_became_active":"2023-10-07T07:58:40.596706+0000","last_became_peered":"2023-10-07T07:58:40.596706+0000","last_unstale":"2023-10-07T07:58:40.604288+0000","last_undegraded":"2023-10-07T07:58:40.604288+0000","last_fullsized":"2023-10-07T07:58:40.604288+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:01.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604178+0000","last_change":"2023-10-07T07:58:40.598661+0000","last_active":"2023-10-07T07:58:40.604178+0000","last_peered":"2023-10-07T07:58:40.604178+0000","last_clean":"2023-10-07T07:58:40.604178+0000","last_became_active":"2023-10-07T07:58:40.598334+0000","last_became_peered":"2023-10-07T07:58:40.598334+0000","last_unstale":"2023-10-07T07:58:40.604178+0000","last_undegraded":"2023-10-07T07:58:40.604178+0000","last_fullsized":"2023-10-07T07:58:40.604178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:04:39.135155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609284+0000","last_change":"2023-10-07T07:58:40.605521+0000","last_active":"2023-10-07T07:58:40.609284+0000","last_peered":"2023-10-07T07:58:40.609284+0000","last_clean":"2023-10-07T07:58:40.609284+0000","last_became_active":"2023-10-07T07:58:40.603261+0000","last_became_peered":"2023-10-07T07:58:40.603261+0000","last_unstale":"2023-10-07T07:58:40.609284+0000","last_undegraded":"2023-10-07T07:58:40.609284+0000","last_fullsized":"2023-10-07T07:58:40.609284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:59:15.735667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.592959+0000","last_change":"2023-10-07T07:58:40.585021+0000","last_active":"2023-10-07T07:58:40.592959+0000","last_peered":"2023-10-07T07:58:40.592959+0000","last_clean":"2023-10-07T07:58:40.592959+0000","last_became_active":"2023-10-07T07:58:40.584726+0000","last_became_peered":"2023-10-07T07:58:40.584726+0000","last_unstale":"2023-10-07T07:58:40.592959+0000","last_undegraded":"2023-10-07T07:58:40.592959+0000","last_fullsized":"2023-10-07T07:58:40.592959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:51:17.036352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.580215+0000","last_change":"2023-10-07T07:58:40.578940+0000","last_active":"2023-10-07T07:58:40.580215+0000","last_peered":"2023-10-07T07:58:40.580215+0000","last_clean":"2023-10-07T07:58:40.580215+0000","last_became_active":"2023-10-07T07:58:40.578651+0000","last_became_peered":"2023-10-07T07:58:40.578651+0000","last_unstale":"2023-10-07T07:58:40.580215+0000","last_undegraded":"2023-10-07T07:58:40.580215+0000","last_fullsized":"2023-10-07T07:58:40.580215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:02:58.823561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.611972+0000","last_change":"2023-10-07T07:58:39.587306+0000","last_active":"2023-10-07T07:58:40.611972+0000","last_peered":"2023-10-07T07:58:40.611972+0000","last_clean":"2023-10-07T07:58:40.611972+0000","last_became_active":"2023-10-07T07:58:39.587000+0000","last_became_peered":"2023-10-07T07:58:39.587000+0000","last_unstale":"2023-10-07T07:58:40.611972+0000","last_undegraded":"2023-10-07T07:58:40.611972+0000","last_fullsized":"2023-10-07T07:58:40.611972+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:02:37.926625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.605736+0000","last_change":"2023-10-07T07:58:40.593542+0000","last_active":"2023-10-07T07:58:40.605736+0000","last_peered":"2023-10-07T07:58:40.605736+0000","last_clean":"2023-10-07T07:58:40.605736+0000","last_became_active":"2023-10-07T07:58:40.589440+0000","last_became_peered":"2023-10-07T07:58:40.589440+0000","last_unstale":"2023-10-07T07:58:40.605736+0000","last_undegraded":"2023-10-07T07:58:40.605736+0000","last_fullsized":"2023-10-07T07:58:40.605736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:36:03.367218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609462+0000","last_change":"2023-10-07T07:58:39.583198+0000","last_active":"2023-10-07T07:58:40.609462+0000","last_peered":"2023-10-07T07:58:40.609462+0000","last_clean":"2023-10-07T07:58:40.609462+0000","last_became_active":"2023-10-07T07:58:39.582404+0000","last_became_peered":"2023-10-07T07:58:39.582404+0000","last_unstale":"2023-10-07T07:58:40.609462+0000","last_undegraded":"2023-10-07T07:58:40.609462+0000","last_fullsized":"2023-10-07T07:58:40.609462+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:58:49.790881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609009+0000","last_change":"2023-10-07T07:58:40.599242+0000","last_active":"2023-10-07T07:58:40.609009+0000","last_peered":"2023-10-07T07:58:40.609009+0000","last_clean":"2023-10-07T07:58:40.609009+0000","last_became_active":"2023-10-07T07:58:40.598952+0000","last_became_peered":"2023-10-07T07:58:40.598952+0000","last_unstale":"2023-10-07T07:58:40.609009+0000","last_undegraded":"2023-10-07T07:58:40.609009+0000","last_fullsized":"2023-10-07T07:58:40.609009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:04:39.135155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.603292+0000","last_change":"2023-10-07T07:58:39.585946+0000","last_active":"2023-10-07T07:58:40.603292+0000","last_peered":"2023-10-07T07:58:40.603292+0000","last_clean":"2023-10-07T07:58:40.603292+0000","last_became_active":"2023-10-07T07:58:39.585658+0000","last_became_peered":"2023-10-07T07:58:39.585658+0000","last_unstale":"2023-10-07T07:58:40.603292+0000","last_undegraded":"2023-10-07T07:58:40.603292+0000","last_fullsized":"2023-10-07T07:58:40.603292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:09:25.972791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.590126+0000","last_change":"2023-10-07T07:58:40.588737+0000","last_active":"2023-10-07T07:58:40.590126+0000","last_peered":"2023-10-07T07:58:40.590126+0000","last_clean":"2023-10-07T07:58:40.590126+0000","last_became_active":"2023-10-07T07:58:40.588160+0000","last_became_peered":"2023-10-07T07:58:40.588160+0000","last_unstale":"2023-10-07T07:58:40.590126+0000","last_undegraded":"2023-10-07T07:58:40.590126+0000","last_fullsized":"2023-10-07T07:58:40.590126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:03.887160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604097+0000","last_change":"2023-10-07T07:58:39.590216+0000","last_active":"2023-10-07T07:58:40.604097+0000","last_peered":"2023-10-07T07:58:40.604097+0000","last_clean":"2023-10-07T07:58:40.604097+0000","last_became_active":"2023-10-07T07:58:39.589842+0000","last_became_peered":"2023-10-07T07:58:39.589842+0000","last_unstale":"2023-10-07T07:58:40.604097+0000","last_undegraded":"2023-10-07T07:58:40.604097+0000","last_fullsized":"2023-10-07T07:58:40.604097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:00:59.089825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.597974+0000","last_change":"2023-10-07T07:58:40.596239+0000","last_active":"2023-10-07T07:58:40.597974+0000","last_peered":"2023-10-07T07:58:40.597974+0000","last_clean":"2023-10-07T07:58:40.597974+0000","last_became_active":"2023-10-07T07:58:40.595905+0000","last_became_peered":"2023-10-07T07:58:40.595905+0000","last_unstale":"2023-10-07T07:58:40.597974+0000","last_undegraded":"2023-10-07T07:58:40.597974+0000","last_fullsized":"2023-10-07T07:58:40.597974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:31.416135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.608691+0000","last_change":"2023-10-07T07:58:39.592109+0000","last_active":"2023-10-07T07:58:40.608691+0000","last_peered":"2023-10-07T07:58:40.608691+0000","last_clean":"2023-10-07T07:58:40.608691+0000","last_became_active":"2023-10-07T07:58:39.591824+0000","last_became_peered":"2023-10-07T07:58:39.591824+0000","last_unstale":"2023-10-07T07:58:40.608691+0000","last_undegraded":"2023-10-07T07:58:40.608691+0000","last_fullsized":"2023-10-07T07:58:40.608691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:02:50.626593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609232+0000","last_change":"2023-10-07T07:58:40.604850+0000","last_active":"2023-10-07T07:58:40.609232+0000","last_peered":"2023-10-07T07:58:40.609232+0000","last_clean":"2023-10-07T07:58:40.609232+0000","last_became_active":"2023-10-07T07:58:40.600203+0000","last_became_peered":"2023-10-07T07:58:40.600203+0000","last_unstale":"2023-10-07T07:58:40.609232+0000","last_undegraded":"2023-10-07T07:58:40.609232+0000","last_fullsized":"2023-10-07T07:58:40.609232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:46:05.776616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.608791+0000","last_change":"2023-10-07T07:58:39.594140+0000","last_active":"2023-10-07T07:58:40.608791+0000","last_peered":"2023-10-07T07:58:40.608791+0000","last_clean":"2023-10-07T07:58:40.608791+0000","last_became_active":"2023-10-07T07:58:39.593773+0000","last_became_peered":"2023-10-07T07:58:39.593773+0000","last_unstale":"2023-10-07T07:58:40.608791+0000","last_undegraded":"2023-10-07T07:58:40.608791+0000","last_fullsized":"2023-10-07T07:58:40.608791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:02:50.444828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.610055+0000","last_change":"2023-10-07T07:58:40.605444+0000","last_active":"2023-10-07T07:58:40.610055+0000","last_peered":"2023-10-07T07:58:40.610055+0000","last_clean":"2023-10-07T07:58:40.610055+0000","last_became_active":"2023-10-07T07:58:40.605135+0000","last_became_peered":"2023-10-07T07:58:40.605135+0000","last_unstale":"2023-10-07T07:58:40.610055+0000","last_undegraded":"2023-10-07T07:58:40.610055+0000","last_fullsized":"2023-10-07T07:58:40.610055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:02:51.450726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.582616+0000","last_change":"2023-10-07T07:58:39.585602+0000","last_active":"2023-10-07T07:58:40.582616+0000","last_peered":"2023-10-07T07:58:40.582616+0000","last_clean":"2023-10-07T07:58:40.582616+0000","last_became_active":"2023-10-07T07:58:39.585291+0000","last_became_peered":"2023-10-07T07:58:39.585291+0000","last_unstale":"2023-10-07T07:58:40.582616+0000","last_undegraded":"2023-10-07T07:58:40.582616+0000","last_fullsized":"2023-10-07T07:58:40.582616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:03.308648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595444+0000","last_change":"2023-10-07T07:58:40.593699+0000","last_active":"2023-10-07T07:58:40.595444+0000","last_peered":"2023-10-07T07:58:40.595444+0000","last_clean":"2023-10-07T07:58:40.595444+0000","last_became_active":"2023-10-07T07:58:40.593452+0000","last_became_peered":"2023-10-07T07:58:40.593452+0000","last_unstale":"2023-10-07T07:58:40.595444+0000","last_undegraded":"2023-10-07T07:58:40.595444+0000","last_fullsized":"2023-10-07T07:58:40.595444+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:04:36.620305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.581824+0000","last_change":"2023-10-07T07:58:39.600505+0000","last_active":"2023-10-07T07:58:40.581824+0000","last_peered":"2023-10-07T07:58:40.581824+0000","last_clean":"2023-10-07T07:58:40.581824+0000","last_became_active":"2023-10-07T07:58:39.600091+0000","last_became_peered":"2023-10-07T07:58:39.600091+0000","last_unstale":"2023-10-07T07:58:40.581824+0000","last_undegraded":"2023-10-07T07:58:40.581824+0000","last_fullsized":"2023-10-07T07:58:40.581824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:03:13.602007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.592599+0000","last_change":"2023-10-07T07:58:40.582833+0000","last_active":"2023-10-07T07:58:40.592599+0000","last_peered":"2023-10-07T07:58:40.592599+0000","last_clean":"2023-10-07T07:58:40.592599+0000","last_became_active":"2023-10-07T07:58:40.582551+0000","last_became_peered":"2023-10-07T07:58:40.582551+0000","last_unstale":"2023-10-07T07:58:40.592599+0000","last_undegraded":"2023-10-07T07:58:40.592599+0000","last_fullsized":"2023-10-07T07:58:40.592599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:10:38.683774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.585774+0000","last_change":"2023-10-07T07:58:39.587730+0000","last_active":"2023-10-07T07:58:40.585774+0000","last_peered":"2023-10-07T07:58:40.585774+0000","last_clean":"2023-10-07T07:58:40.585774+0000","last_became_active":"2023-10-07T07:58:39.587461+0000","last_became_peered":"2023-10-07T07:58:39.587461+0000","last_unstale":"2023-10-07T07:58:40.585774+0000","last_undegraded":"2023-10-07T07:58:40.585774+0000","last_fullsized":"2023-10-07T07:58:40.585774+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:36:02.361320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.592212+0000","last_change":"2023-10-07T07:58:40.589841+0000","last_active":"2023-10-07T07:58:40.592212+0000","last_peered":"2023-10-07T07:58:40.592212+0000","last_clean":"2023-10-07T07:58:40.592212+0000","last_became_active":"2023-10-07T07:58:40.589503+0000","last_became_peered":"2023-10-07T07:58:40.589503+0000","last_unstale":"2023-10-07T07:58:40.592212+0000","last_undegraded":"2023-10-07T07:58:40.592212+0000","last_fullsized":"2023-10-07T07:58:40.592212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:31:14.816056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.585727+0000","last_change":"2023-10-07T07:58:39.586723+0000","last_active":"2023-10-07T07:58:40.585727+0000","last_peered":"2023-10-07T07:58:40.585727+0000","last_clean":"2023-10-07T07:58:40.585727+0000","last_became_active":"2023-10-07T07:58:39.586265+0000","last_became_peered":"2023-10-07T07:58:39.586265+0000","last_unstale":"2023-10-07T07:58:40.585727+0000","last_undegraded":"2023-10-07T07:58:40.585727+0000","last_fullsized":"2023-10-07T07:58:40.585727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:08:19.384290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595467+0000","last_change":"2023-10-07T07:58:40.592828+0000","last_active":"2023-10-07T07:58:40.595467+0000","last_peered":"2023-10-07T07:58:40.595467+0000","last_clean":"2023-10-07T07:58:40.595467+0000","last_became_active":"2023-10-07T07:58:40.590497+0000","last_became_peered":"2023-10-07T07:58:40.590497+0000","last_unstale":"2023-10-07T07:58:40.595467+0000","last_undegraded":"2023-10-07T07:58:40.595467+0000","last_fullsized":"2023-10-07T07:58:40.595467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:06:51.002473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.611831+0000","last_change":"2023-10-07T07:58:39.591656+0000","last_active":"2023-10-07T07:58:40.611831+0000","last_peered":"2023-10-07T07:58:40.611831+0000","last_clean":"2023-10-07T07:58:40.611831+0000","last_became_active":"2023-10-07T07:58:39.588856+0000","last_became_peered":"2023-10-07T07:58:39.588856+0000","last_unstale":"2023-10-07T07:58:40.611831+0000","last_undegraded":"2023-10-07T07:58:40.611831+0000","last_fullsized":"2023-10-07T07:58:40.611831+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:31:13.810157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609792+0000","last_change":"2023-10-07T07:58:40.589887+0000","last_active":"2023-10-07T07:58:40.609792+0000","last_peered":"2023-10-07T07:58:40.609792+0000","last_clean":"2023-10-07T07:58:40.609792+0000","last_became_active":"2023-10-07T07:58:40.589590+0000","last_became_peered":"2023-10-07T07:58:40.589590+0000","last_unstale":"2023-10-07T07:58:40.609792+0000","last_undegraded":"2023-10-07T07:58:40.609792+0000","last_fullsized":"2023-10-07T07:58:40.609792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:17:19.907560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.610739+0000","last_change":"2023-10-07T07:58:39.583848+0000","last_active":"2023-10-07T07:58:40.610739+0000","last_peered":"2023-10-07T07:58:40.610739+0000","last_clean":"2023-10-07T07:58:40.610739+0000","last_became_active":"2023-10-07T07:58:39.583137+0000","last_became_peered":"2023-10-07T07:58:39.583137+0000","last_unstale":"2023-10-07T07:58:40.610739+0000","last_undegraded":"2023-10-07T07:58:40.610739+0000","last_fullsized":"2023-10-07T07:58:40.610739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:30.410237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602551+0000","last_change":"2023-10-07T07:58:40.597566+0000","last_active":"2023-10-07T07:58:40.602551+0000","last_peered":"2023-10-07T07:58:40.602551+0000","last_clean":"2023-10-07T07:58:40.602551+0000","last_became_active":"2023-10-07T07:58:40.597266+0000","last_became_peered":"2023-10-07T07:58:40.597266+0000","last_unstale":"2023-10-07T07:58:40.602551+0000","last_undegraded":"2023-10-07T07:58:40.602551+0000","last_fullsized":"2023-10-07T07:58:40.602551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:29:17.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609173+0000","last_change":"2023-10-07T07:58:39.593903+0000","last_active":"2023-10-07T07:58:40.609173+0000","last_peered":"2023-10-07T07:58:40.609173+0000","last_clean":"2023-10-07T07:58:40.609173+0000","last_became_active":"2023-10-07T07:58:39.593607+0000","last_became_peered":"2023-10-07T07:58:39.593607+0000","last_unstale":"2023-10-07T07:58:40.609173+0000","last_undegraded":"2023-10-07T07:58:40.609173+0000","last_fullsized":"2023-10-07T07:58:40.609173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:56:27.881180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.590170+0000","last_change":"2023-10-07T07:58:40.588847+0000","last_active":"2023-10-07T07:58:40.590170+0000","last_peered":"2023-10-07T07:58:40.590170+0000","last_clean":"2023-10-07T07:58:40.590170+0000","last_became_active":"2023-10-07T07:58:40.588336+0000","last_became_peered":"2023-10-07T07:58:40.588336+0000","last_unstale":"2023-10-07T07:58:40.590170+0000","last_undegraded":"2023-10-07T07:58:40.590170+0000","last_fullsized":"2023-10-07T07:58:40.590170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:21:39.170465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.580148+0000","last_change":"2023-10-07T07:58:39.584100+0000","last_active":"2023-10-07T07:58:40.580148+0000","last_peered":"2023-10-07T07:58:40.580148+0000","last_clean":"2023-10-07T07:58:40.580148+0000","last_became_active":"2023-10-07T07:58:39.583747+0000","last_became_peered":"2023-10-07T07:58:39.583747+0000","last_unstale":"2023-10-07T07:58:40.580148+0000","last_undegraded":"2023-10-07T07:58:40.580148+0000","last_fullsized":"2023-10-07T07:58:40.580148+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:44.993394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609294+0000","last_change":"2023-10-07T07:58:40.603617+0000","last_active":"2023-10-07T07:58:40.609294+0000","last_peered":"2023-10-07T07:58:40.609294+0000","last_clean":"2023-10-07T07:58:40.609294+0000","last_became_active":"2023-10-07T07:58:40.599942+0000","last_became_peered":"2023-10-07T07:58:40.599942+0000","last_unstale":"2023-10-07T07:58:40.609294+0000","last_undegraded":"2023-10-07T07:58:40.609294+0000","last_fullsized":"2023-10-07T07:58:40.609294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:29:17.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.578841+0000","last_change":"2023-10-07T07:58:39.586423+0000","last_active":"2023-10-07T07:58:40.578841+0000","last_peered":"2023-10-07T07:58:40.578841+0000","last_clean":"2023-10-07T07:58:40.578841+0000","last_became_active":"2023-10-07T07:58:39.586121+0000","last_became_peered":"2023-10-07T07:58:39.586121+0000","last_unstale":"2023-10-07T07:58:40.578841+0000","last_undegraded":"2023-10-07T07:58:40.578841+0000","last_fullsized":"2023-10-07T07:58:40.578841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:39:27.575737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.605801+0000","last_change":"2023-10-07T07:58:40.588568+0000","last_active":"2023-10-07T07:58:40.605801+0000","last_peered":"2023-10-07T07:58:40.605801+0000","last_clean":"2023-10-07T07:58:40.605801+0000","last_became_active":"2023-10-07T07:58:40.587823+0000","last_became_peered":"2023-10-07T07:58:40.587823+0000","last_unstale":"2023-10-07T07:58:40.605801+0000","last_undegraded":"2023-10-07T07:58:40.605801+0000","last_fullsized":"2023-10-07T07:58:40.605801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:35:36.270143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.611870+0000","last_change":"2023-10-07T07:58:39.585927+0000","last_active":"2023-10-07T07:58:40.611870+0000","last_peered":"2023-10-07T07:58:40.611870+0000","last_clean":"2023-10-07T07:58:40.611870+0000","last_became_active":"2023-10-07T07:58:39.585622+0000","last_became_peered":"2023-10-07T07:58:39.585622+0000","last_unstale":"2023-10-07T07:58:40.611870+0000","last_undegraded":"2023-10-07T07:58:40.611870+0000","last_fullsized":"2023-10-07T07:58:40.611870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:26:31.499283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.603981+0000","last_change":"2023-10-07T07:58:40.588526+0000","last_active":"2023-10-07T07:58:40.603981+0000","last_peered":"2023-10-07T07:58:40.603981+0000","last_clean":"2023-10-07T07:58:40.603981+0000","last_became_active":"2023-10-07T07:58:40.588208+0000","last_became_peered":"2023-10-07T07:58:40.588208+0000","last_unstale":"2023-10-07T07:58:40.603981+0000","last_undegraded":"2023-10-07T07:58:40.603981+0000","last_fullsized":"2023-10-07T07:58:40.603981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:25:55.442476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.603237+0000","last_change":"2023-10-07T07:58:39.588571+0000","last_active":"2023-10-07T07:58:40.603237+0000","last_peered":"2023-10-07T07:58:40.603237+0000","last_clean":"2023-10-07T07:58:40.603237+0000","last_became_active":"2023-10-07T07:58:39.588297+0000","last_became_peered":"2023-10-07T07:58:39.588297+0000","last_unstale":"2023-10-07T07:58:40.603237+0000","last_undegraded":"2023-10-07T07:58:40.603237+0000","last_fullsized":"2023-10-07T07:58:40.603237+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:34:00.214894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.610120+0000","last_change":"2023-10-07T07:58:40.606422+0000","last_active":"2023-10-07T07:58:40.610120+0000","last_peered":"2023-10-07T07:58:40.610120+0000","last_clean":"2023-10-07T07:58:40.610120+0000","last_became_active":"2023-10-07T07:58:40.605829+0000","last_became_peered":"2023-10-07T07:58:40.605829+0000","last_unstale":"2023-10-07T07:58:40.610120+0000","last_undegraded":"2023-10-07T07:58:40.610120+0000","last_fullsized":"2023-10-07T07:58:40.610120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:10:32.121446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.608766+0000","last_change":"2023-10-07T07:58:39.594216+0000","last_active":"2023-10-07T07:58:40.608766+0000","last_peered":"2023-10-07T07:58:40.608766+0000","last_clean":"2023-10-07T07:58:40.608766+0000","last_became_active":"2023-10-07T07:58:39.593897+0000","last_became_peered":"2023-10-07T07:58:39.593897+0000","last_unstale":"2023-10-07T07:58:40.608766+0000","last_undegraded":"2023-10-07T07:58:40.608766+0000","last_fullsized":"2023-10-07T07:58:40.608766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:10:31.115548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602822+0000","last_change":"2023-10-07T07:58:40.598706+0000","last_active":"2023-10-07T07:58:40.602822+0000","last_peered":"2023-10-07T07:58:40.602822+0000","last_clean":"2023-10-07T07:58:40.602822+0000","last_became_active":"2023-10-07T07:58:40.598438+0000","last_became_peered":"2023-10-07T07:58:40.598438+0000","last_unstale":"2023-10-07T07:58:40.602822+0000","last_undegraded":"2023-10-07T07:58:40.602822+0000","last_fullsized":"2023-10-07T07:58:40.602822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:03.887160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.611347+0000","last_change":"2023-10-07T07:58:39.591836+0000","last_active":"2023-10-07T07:58:40.611347+0000","last_peered":"2023-10-07T07:58:40.611347+0000","last_clean":"2023-10-07T07:58:40.611347+0000","last_became_active":"2023-10-07T07:58:39.590987+0000","last_became_peered":"2023-10-07T07:58:39.590987+0000","last_unstale":"2023-10-07T07:58:40.611347+0000","last_undegraded":"2023-10-07T07:58:40.611347+0000","last_fullsized":"2023-10-07T07:58:40.611347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:04:35.614406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602856+0000","last_change":"2023-10-07T07:58:40.599435+0000","last_active":"2023-10-07T07:58:40.602856+0000","last_peered":"2023-10-07T07:58:40.602856+0000","last_clean":"2023-10-07T07:58:40.602856+0000","last_became_active":"2023-10-07T07:58:40.599166+0000","last_became_peered":"2023-10-07T07:58:40.599166+0000","last_unstale":"2023-10-07T07:58:40.602856+0000","last_undegraded":"2023-10-07T07:58:40.602856+0000","last_fullsized":"2023-10-07T07:58:40.602856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:01:00.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.585520+0000","last_change":"2023-10-07T07:58:39.590387+0000","last_active":"2023-10-07T07:58:40.585520+0000","last_peered":"2023-10-07T07:58:40.585520+0000","last_clean":"2023-10-07T07:58:40.585520+0000","last_became_active":"2023-10-07T07:58:39.590051+0000","last_became_peered":"2023-10-07T07:58:39.590051+0000","last_unstale":"2023-10-07T07:58:40.585520+0000","last_undegraded":"2023-10-07T07:58:40.585520+0000","last_fullsized":"2023-10-07T07:58:40.585520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:10:36.901386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.596736+0000","last_change":"2023-10-07T07:58:40.591574+0000","last_active":"2023-10-07T07:58:40.596736+0000","last_peered":"2023-10-07T07:58:40.596736+0000","last_clean":"2023-10-07T07:58:40.596736+0000","last_became_active":"2023-10-07T07:58:40.591330+0000","last_became_peered":"2023-10-07T07:58:40.591330+0000","last_unstale":"2023-10-07T07:58:40.596736+0000","last_undegraded":"2023-10-07T07:58:40.596736+0000","last_fullsized":"2023-10-07T07:58:40.596736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:59:15.735667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604621+0000","last_change":"2023-10-07T07:58:39.590861+0000","last_active":"2023-10-07T07:58:40.604621+0000","last_peered":"2023-10-07T07:58:40.604621+0000","last_clean":"2023-10-07T07:58:40.604621+0000","last_became_active":"2023-10-07T07:58:39.590587+0000","last_became_peered":"2023-10-07T07:58:39.590587+0000","last_unstale":"2023-10-07T07:58:40.604621+0000","last_undegraded":"2023-10-07T07:58:40.604621+0000","last_fullsized":"2023-10-07T07:58:40.604621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:34:15.142970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.597099+0000","last_change":"2023-10-07T07:58:40.594675+0000","last_active":"2023-10-07T07:58:40.597099+0000","last_peered":"2023-10-07T07:58:40.597099+0000","last_clean":"2023-10-07T07:58:40.597099+0000","last_became_active":"2023-10-07T07:58:40.592821+0000","last_became_peered":"2023-10-07T07:58:40.592821+0000","last_unstale":"2023-10-07T07:58:40.597099+0000","last_undegraded":"2023-10-07T07:58:40.597099+0000","last_fullsized":"2023-10-07T07:58:40.597099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:14.250170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.585573+0000","last_change":"2023-10-07T07:58:39.594744+0000","last_active":"2023-10-07T07:58:40.585573+0000","last_peered":"2023-10-07T07:58:40.585573+0000","last_clean":"2023-10-07T07:58:40.585573+0000","last_became_active":"2023-10-07T07:58:39.594470+0000","last_became_peered":"2023-10-07T07:58:39.594470+0000","last_unstale":"2023-10-07T07:58:40.585573+0000","last_undegraded":"2023-10-07T07:58:40.585573+0000","last_fullsized":"2023-10-07T07:58:40.585573+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:00:59.089825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.590635+0000","last_change":"2023-10-07T07:58:40.587272+0000","last_active":"2023-10-07T07:58:40.590635+0000","last_peered":"2023-10-07T07:58:40.590635+0000","last_clean":"2023-10-07T07:58:40.590635+0000","last_became_active":"2023-10-07T07:58:40.586960+0000","last_became_peered":"2023-10-07T07:58:40.586960+0000","last_unstale":"2023-10-07T07:58:40.590635+0000","last_undegraded":"2023-10-07T07:58:40.590635+0000","last_fullsized":"2023-10-07T07:58:40.590635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:08:20.390188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.587107+0000","last_change":"2023-10-07T07:58:39.585889+0000","last_active":"2023-10-07T07:58:40.587107+0000","last_peered":"2023-10-07T07:58:40.587107+0000","last_clean":"2023-10-07T07:58:40.587107+0000","last_became_active":"2023-10-07T07:58:39.585598+0000","last_became_peered":"2023-10-07T07:58:39.585598+0000","last_unstale":"2023-10-07T07:58:40.587107+0000","last_undegraded":"2023-10-07T07:58:40.587107+0000","last_fullsized":"2023-10-07T07:58:40.587107+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:04:55.540544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.610160+0000","last_change":"2023-10-07T07:58:40.607004+0000","last_active":"2023-10-07T07:58:40.610160+0000","last_peered":"2023-10-07T07:58:40.610160+0000","last_clean":"2023-10-07T07:58:40.610160+0000","last_became_active":"2023-10-07T07:58:40.606412+0000","last_became_peered":"2023-10-07T07:58:40.606412+0000","last_unstale":"2023-10-07T07:58:40.610160+0000","last_undegraded":"2023-10-07T07:58:40.610160+0000","last_fullsized":"2023-10-07T07:58:40.610160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:01:00.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.576238+0000","last_change":"2023-10-07T07:58:39.582581+0000","last_active":"2023-10-07T07:58:40.576238+0000","last_peered":"2023-10-07T07:58:40.576238+0000","last_clean":"2023-10-07T07:58:40.576238+0000","last_became_active":"2023-10-07T07:58:39.582271+0000","last_became_peered":"2023-10-07T07:58:39.582271+0000","last_unstale":"2023-10-07T07:58:40.576238+0000","last_undegraded":"2023-10-07T07:58:40.576238+0000","last_fullsized":"2023-10-07T07:58:40.576238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:02.881262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602970+0000","last_change":"2023-10-07T07:58:40.599707+0000","last_active":"2023-10-07T07:58:40.602970+0000","last_peered":"2023-10-07T07:58:40.602970+0000","last_clean":"2023-10-07T07:58:40.602970+0000","last_became_active":"2023-10-07T07:58:40.598990+0000","last_became_peered":"2023-10-07T07:58:40.598990+0000","last_unstale":"2023-10-07T07:58:40.602970+0000","last_undegraded":"2023-10-07T07:58:40.602970+0000","last_fullsized":"2023-10-07T07:58:40.602970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:02:51.450726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.607930+0000","last_change":"2023-10-07T07:58:39.584540+0000","last_active":"2023-10-07T07:58:40.607930+0000","last_peered":"2023-10-07T07:58:40.607930+0000","last_clean":"2023-10-07T07:58:40.607930+0000","last_became_active":"2023-10-07T07:58:39.584195+0000","last_became_peered":"2023-10-07T07:58:39.584195+0000","last_unstale":"2023-10-07T07:58:40.607930+0000","last_undegraded":"2023-10-07T07:58:40.607930+0000","last_fullsized":"2023-10-07T07:58:40.607930+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:00:59.089825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604983+0000","last_change":"2023-10-07T07:58:40.592731+0000","last_active":"2023-10-07T07:58:40.604983+0000","last_peered":"2023-10-07T07:58:40.604983+0000","last_clean":"2023-10-07T07:58:40.604983+0000","last_became_active":"2023-10-07T07:58:40.592468+0000","last_became_peered":"2023-10-07T07:58:40.592468+0000","last_unstale":"2023-10-07T07:58:40.604983+0000","last_undegraded":"2023-10-07T07:58:40.604983+0000","last_fullsized":"2023-10-07T07:58:40.604983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:24.940558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-07T07:58:41.719772+0000","last_change":"2023-10-07T07:58:39.586541+0000","last_active":"2023-10-07T07:58:41.719772+0000","last_peered":"2023-10-07T07:58:41.719772+0000","last_clean":"2023-10-07T07:58:41.719772+0000","last_became_active":"2023-10-07T07:58:39.585995+0000","last_became_peered":"2023-10-07T07:58:39.585995+0000","last_unstale":"2023-10-07T07:58:41.719772+0000","last_undegraded":"2023-10-07T07:58:41.719772+0000","last_fullsized":"2023-10-07T07:58:41.719772+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:46:04.770718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600053+0000","last_change":"2023-10-07T07:58:40.592532+0000","last_active":"2023-10-07T07:58:40.600053+0000","last_peered":"2023-10-07T07:58:40.600053+0000","last_clean":"2023-10-07T07:58:40.600053+0000","last_became_active":"2023-10-07T07:58:40.592198+0000","last_became_peered":"2023-10-07T07:58:40.592198+0000","last_unstale":"2023-10-07T07:58:40.600053+0000","last_undegraded":"2023-10-07T07:58:40.600053+0000","last_fullsized":"2023-10-07T07:58:40.600053+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:58:50.796780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.577431+0000","last_change":"2023-10-07T07:58:39.587210+0000","last_active":"2023-10-07T07:58:40.577431+0000","last_peered":"2023-10-07T07:58:40.577431+0000","last_clean":"2023-10-07T07:58:40.577431+0000","last_became_active":"2023-10-07T07:58:39.586827+0000","last_became_peered":"2023-10-07T07:58:39.586827+0000","last_unstale":"2023-10-07T07:58:40.577431+0000","last_undegraded":"2023-10-07T07:58:40.577431+0000","last_fullsized":"2023-10-07T07:58:40.577431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:44.993394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604080+0000","last_change":"2023-10-07T07:58:40.593651+0000","last_active":"2023-10-07T07:58:40.604080+0000","last_peered":"2023-10-07T07:58:40.604080+0000","last_clean":"2023-10-07T07:58:40.604080+0000","last_became_active":"2023-10-07T07:58:40.593377+0000","last_became_peered":"2023-10-07T07:58:40.593377+0000","last_unstale":"2023-10-07T07:58:40.604080+0000","last_undegraded":"2023-10-07T07:58:40.604080+0000","last_fullsized":"2023-10-07T07:58:40.604080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:59:15.735667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.576907+0000","last_change":"2023-10-07T07:58:39.584604+0000","last_active":"2023-10-07T07:58:40.576907+0000","last_peered":"2023-10-07T07:58:40.576907+0000","last_clean":"2023-10-07T07:58:40.576907+0000","last_became_active":"2023-10-07T07:58:39.584209+0000","last_became_peered":"2023-10-07T07:58:39.584209+0000","last_unstale":"2023-10-07T07:58:40.576907+0000","last_undegraded":"2023-10-07T07:58:40.576907+0000","last_fullsized":"2023-10-07T07:58:40.576907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:10:31.115548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595075+0000","last_change":"2023-10-07T07:58:40.592093+0000","last_active":"2023-10-07T07:58:40.595075+0000","last_peered":"2023-10-07T07:58:40.595075+0000","last_clean":"2023-10-07T07:58:40.595075+0000","last_became_active":"2023-10-07T07:58:40.591769+0000","last_became_peered":"2023-10-07T07:58:40.591769+0000","last_unstale":"2023-10-07T07:58:40.595075+0000","last_undegraded":"2023-10-07T07:58:40.595075+0000","last_fullsized":"2023-10-07T07:58:40.595075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:02:25.944882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.576988+0000","last_change":"2023-10-07T07:58:39.584951+0000","last_active":"2023-10-07T07:58:40.576988+0000","last_peered":"2023-10-07T07:58:40.576988+0000","last_clean":"2023-10-07T07:58:40.576988+0000","last_became_active":"2023-10-07T07:58:39.584708+0000","last_became_peered":"2023-10-07T07:58:39.584708+0000","last_unstale":"2023-10-07T07:58:40.576988+0000","last_undegraded":"2023-10-07T07:58:40.576988+0000","last_fullsized":"2023-10-07T07:58:40.576988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:02:50.444828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.594422+0000","last_change":"2023-10-07T07:58:40.590753+0000","last_active":"2023-10-07T07:58:40.594422+0000","last_peered":"2023-10-07T07:58:40.594422+0000","last_clean":"2023-10-07T07:58:40.594422+0000","last_became_active":"2023-10-07T07:58:40.590475+0000","last_became_peered":"2023-10-07T07:58:40.590475+0000","last_unstale":"2023-10-07T07:58:40.594422+0000","last_undegraded":"2023-10-07T07:58:40.594422+0000","last_fullsized":"2023-10-07T07:58:40.594422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:14:18.503786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.583970+0000","last_change":"2023-10-07T07:58:39.593877+0000","last_active":"2023-10-07T07:58:40.583970+0000","last_peered":"2023-10-07T07:58:40.583970+0000","last_clean":"2023-10-07T07:58:40.583970+0000","last_became_active":"2023-10-07T07:58:39.593573+0000","last_became_peered":"2023-10-07T07:58:39.593573+0000","last_unstale":"2023-10-07T07:58:40.583970+0000","last_undegraded":"2023-10-07T07:58:40.583970+0000","last_fullsized":"2023-10-07T07:58:40.583970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:30.410237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.605823+0000","last_change":"2023-10-07T07:58:40.588788+0000","last_active":"2023-10-07T07:58:40.605823+0000","last_peered":"2023-10-07T07:58:40.605823+0000","last_clean":"2023-10-07T07:58:40.605823+0000","last_became_active":"2023-10-07T07:58:40.588280+0000","last_became_peered":"2023-10-07T07:58:40.588280+0000","last_unstale":"2023-10-07T07:58:40.605823+0000","last_undegraded":"2023-10-07T07:58:40.605823+0000","last_fullsized":"2023-10-07T07:58:40.605823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:50:18.856129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.579125+0000","last_change":"2023-10-07T07:58:39.583131+0000","last_active":"2023-10-07T07:58:40.579125+0000","last_peered":"2023-10-07T07:58:40.579125+0000","last_clean":"2023-10-07T07:58:40.579125+0000","last_became_active":"2023-10-07T07:58:39.582847+0000","last_became_peered":"2023-10-07T07:58:39.582847+0000","last_unstale":"2023-10-07T07:58:40.579125+0000","last_undegraded":"2023-10-07T07:58:40.579125+0000","last_fullsized":"2023-10-07T07:58:40.579125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:40:32.472195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604017+0000","last_change":"2023-10-07T07:58:40.593610+0000","last_active":"2023-10-07T07:58:40.604017+0000","last_peered":"2023-10-07T07:58:40.604017+0000","last_clean":"2023-10-07T07:58:40.604017+0000","last_became_active":"2023-10-07T07:58:40.593257+0000","last_became_peered":"2023-10-07T07:58:40.593257+0000","last_unstale":"2023-10-07T07:58:40.604017+0000","last_undegraded":"2023-10-07T07:58:40.604017+0000","last_fullsized":"2023-10-07T07:58:40.604017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:29:17.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.576673+0000","last_change":"2023-10-07T07:58:39.592025+0000","last_active":"2023-10-07T07:58:40.576673+0000","last_peered":"2023-10-07T07:58:40.576673+0000","last_clean":"2023-10-07T07:58:40.576673+0000","last_became_active":"2023-10-07T07:58:39.591387+0000","last_became_peered":"2023-10-07T07:58:39.591387+0000","last_unstale":"2023-10-07T07:58:40.576673+0000","last_undegraded":"2023-10-07T07:58:40.576673+0000","last_fullsized":"2023-10-07T07:58:40.576673+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:00.660419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.588505+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T07:58:40.588505+0000","last_peered":"2023-10-07T07:58:40.588505+0000","last_clean":"2023-10-07T07:58:40.588505+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T07:58:40.588505+0000","last_undegraded":"2023-10-07T07:58:40.588505+0000","last_fullsized":"2023-10-07T07:58:40.588505+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":143706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":138932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":734122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":143774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":143718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.372}]},{"osd":1,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.40200000000000002}]},{"osd":4,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71099999999999997}]},{"osd":5,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72499999999999998}]},{"osd":7,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.79700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":144432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":138954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51500000000000001}]},{"osd":1,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.67100000000000004}]},{"osd":6,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.752}]},{"osd":7,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":734606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T07:58:44.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T07:58:44.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.942+0000 7fbf9f9f4700 1 Processor -- start 2023-10-07T07:58:44.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.942+0000 7fbf9f9f4700 1 -- start start 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9f9f4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 0x7fbf980c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9f9f4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf980ced30 0x7fbf980c3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9f9f4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf981596e0 0x7fbf9806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9f9f4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf98068100 con 0x7fbf981596e0 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9f9f4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf980c36c0 con 0x7fbf980ced30 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9f9f4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf98068350 con 0x7fbf98159310 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9df91700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf981596e0 0x7fbf9806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9df91700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf981596e0 0x7fbf9806b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56360/0 (socket says 172.21.15.7:56360) 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9df91700 1 -- 172.21.15.7:0/1923432888 learned_addr learned my addr 172.21.15.7:0/1923432888 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9d790700 1 --2- 172.21.15.7:0/1923432888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 0x7fbf980c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9cf8f700 1 --2- 172.21.15.7:0/1923432888 >> v2:172.21.15.172:3300/0 conn(0x7fbf980ced30 0x7fbf980c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9df91700 1 -- 172.21.15.7:0/1923432888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 msgr2=0x7fbf980c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9df91700 1 --2- 172.21.15.7:0/1923432888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 0x7fbf980c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9df91700 1 -- 172.21.15.7:0/1923432888 >> v2:172.21.15.172:3300/0 conn(0x7fbf980ced30 msgr2=0x7fbf980c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9df91700 1 --2- 172.21.15.7:0/1923432888 >> v2:172.21.15.172:3300/0 conn(0x7fbf980ced30 0x7fbf980c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9df91700 1 -- 172.21.15.7:0/1923432888 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf98068630 con 0x7fbf981596e0 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.943+0000 7fbf9cf8f700 1 --2- 172.21.15.7:0/1923432888 >> v2:172.21.15.172:3300/0 conn(0x7fbf980ced30 0x7fbf980c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf9df91700 1 --2- 172.21.15.7:0/1923432888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf981596e0 0x7fbf9806b620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009f20 tx=0x7fbf90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54026c3286127ade server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf8ffff700 1 -- 172.21.15.7:0/1923432888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000e070 con 0x7fbf981596e0 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf8ffff700 1 -- 172.21.15.7:0/1923432888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9000b990 con 0x7fbf981596e0 2023-10-07T07:58:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf8ffff700 1 -- 172.21.15.7:0/1923432888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900092e0 con 0x7fbf981596e0 2023-10-07T07:58:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/1923432888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf981596e0 msgr2=0x7fbf9806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/1923432888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf981596e0 0x7fbf9806b620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009f20 tx=0x7fbf90009840 comp rx=0 tx=0).stop 2023-10-07T07:58:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/1923432888 shutdown_connections 2023-10-07T07:58:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/1923432888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf981596e0 0x7fbf9806b620 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/1923432888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 0x7fbf980c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/1923432888 >> v2:172.21.15.172:3300/0 conn(0x7fbf980ced30 0x7fbf980c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.944+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/1923432888 >> 172.21.15.7:0/1923432888 conn(0x7fbf98067ac0 msgr2=0x7fbf980ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.945+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/1923432888 shutdown_connections 2023-10-07T07:58:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.945+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/1923432888 wait complete. 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.946+0000 7fbf9f9f4700 1 Processor -- start 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.946+0000 7fbf9f9f4700 1 -- start start 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.947+0000 7fbf9f9f4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 0x7fbf9813bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.947+0000 7fbf9f9f4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf981596e0 0x7fbf98157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.947+0000 7fbf9f9f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf9813a940 0x7fbf9813adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.947+0000 7fbf9f9f4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9813b300 con 0x7fbf9813a940 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.947+0000 7fbf9f9f4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf980c5b30 con 0x7fbf981596e0 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.947+0000 7fbf9f9f4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf980c5db0 con 0x7fbf98159310 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9df91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf9813a940 0x7fbf9813adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9df91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf9813a940 0x7fbf9813adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56376/0 (socket says 172.21.15.7:56376) 2023-10-07T07:58:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9df91700 1 -- 172.21.15.7:0/2651914830 learned_addr learned my addr 172.21.15.7:0/2651914830 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9cf8f700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf981596e0 0x7fbf98157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9d790700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 0x7fbf9813bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9df91700 1 -- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 msgr2=0x7fbf9813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9df91700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 0x7fbf9813bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9df91700 1 -- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf981596e0 msgr2=0x7fbf98157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9df91700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf981596e0 0x7fbf98157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9df91700 1 -- 172.21.15.7:0/2651914830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90009ae0 con 0x7fbf9813a940 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9df91700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf9813a940 0x7fbf9813adc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf90012000 tx=0x7fbf90018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf8d7fa700 1 -- 172.21.15.7:0/2651914830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90008ad0 con 0x7fbf9813a940 2023-10-07T07:58:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf8d7fa700 1 -- 172.21.15.7:0/2651914830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf90008c70 con 0x7fbf9813a940 2023-10-07T07:58:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf980c6090 con 0x7fbf9813a940 2023-10-07T07:58:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf8d7fa700 1 -- 172.21.15.7:0/2651914830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90026460 con 0x7fbf9813a940 2023-10-07T07:58:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.948+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf980c6650 con 0x7fbf9813a940 2023-10-07T07:58:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.949+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf980716c0 con 0x7fbf9813a940 2023-10-07T07:58:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.950+0000 7fbf8d7fa700 1 -- 172.21.15.7:0/2651914830 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf90003040 con 0x7fbf9813a940 2023-10-07T07:58:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.950+0000 7fbf8d7fa700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbf7c066f30 0x7fbf7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.951+0000 7fbf8d7fa700 1 -- 172.21.15.7:0/2651914830 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbf90086340 con 0x7fbf9813a940 2023-10-07T07:58:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.951+0000 7fbf9d790700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbf7c066f30 0x7fbf7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.951+0000 7fbf9d790700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbf7c066f30 0x7fbf7c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009d10 tx=0x7fbf940091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:44.955+0000 7fbf8d7fa700 1 -- 172.21.15.7:0/2651914830 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf900535b0 con 0x7fbf9813a940 2023-10-07T07:58:45.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.152+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf980cea10 con 0x7fbf7c066f30 2023-10-07T07:58:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.161+0000 7fbf8d7fa700 1 -- 172.21.15.7:0/2651914830 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271074 (secure 0 0 0) 0x7fbf980cea10 con 0x7fbf7c066f30 2023-10-07T07:58:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbf7c066f30 msgr2=0x7fbf7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbf7c066f30 0x7fbf7c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009d10 tx=0x7fbf940091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf9813a940 msgr2=0x7fbf9813adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf9813a940 0x7fbf9813adc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf90012000 tx=0x7fbf90018000 comp rx=0 tx=0).stop 2023-10-07T07:58:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 shutdown_connections 2023-10-07T07:58:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbf7c066f30 0x7fbf7c0693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf9813a940 0x7fbf9813adc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf98159310 0x7fbf9813bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 --2- 172.21.15.7:0/2651914830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf981596e0 0x7fbf98157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 >> 172.21.15.7:0/2651914830 conn(0x7fbf98067ac0 msgr2=0x7fbf980cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 shutdown_connections 2023-10-07T07:58:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.164+0000 7fbf9f9f4700 1 -- 172.21.15.7:0/2651914830 wait complete. 2023-10-07T07:58:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T07:58:45.178 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2023-10-07T07:58:44.037487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382416,"kb_used_data":4112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592304,"statfs":{"total":773094113280,"available":770654519296,"internally_reserved":0,"allocated":4210688,"data_stored":2322244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.474125"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609287+0000","last_change":"2023-10-07T07:58:40.606566+0000","last_active":"2023-10-07T07:58:40.609287+0000","last_peered":"2023-10-07T07:58:40.609287+0000","last_clean":"2023-10-07T07:58:40.609287+0000","last_became_active":"2023-10-07T07:58:40.606336+0000","last_became_peered":"2023-10-07T07:58:40.606336+0000","last_unstale":"2023-10-07T07:58:40.609287+0000","last_undegraded":"2023-10-07T07:58:40.609287+0000","last_fullsized":"2023-10-07T07:58:40.609287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:31:14.816056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.597711+0000","last_change":"2023-10-07T07:58:40.593477+0000","last_active":"2023-10-07T07:58:40.597711+0000","last_peered":"2023-10-07T07:58:40.597711+0000","last_clean":"2023-10-07T07:58:40.597711+0000","last_became_active":"2023-10-07T07:58:40.593184+0000","last_became_peered":"2023-10-07T07:58:40.593184+0000","last_unstale":"2023-10-07T07:58:40.597711+0000","last_undegraded":"2023-10-07T07:58:40.597711+0000","last_fullsized":"2023-10-07T07:58:40.597711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:01.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600124+0000","last_change":"2023-10-07T07:58:40.596336+0000","last_active":"2023-10-07T07:58:40.600124+0000","last_peered":"2023-10-07T07:58:40.600124+0000","last_clean":"2023-10-07T07:58:40.600124+0000","last_became_active":"2023-10-07T07:58:40.595994+0000","last_became_peered":"2023-10-07T07:58:40.595994+0000","last_unstale":"2023-10-07T07:58:40.600124+0000","last_undegraded":"2023-10-07T07:58:40.600124+0000","last_fullsized":"2023-10-07T07:58:40.600124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:39:28.581635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595449+0000","last_change":"2023-10-07T07:58:40.592783+0000","last_active":"2023-10-07T07:58:40.595449+0000","last_peered":"2023-10-07T07:58:40.595449+0000","last_clean":"2023-10-07T07:58:40.595449+0000","last_became_active":"2023-10-07T07:58:40.592512+0000","last_became_peered":"2023-10-07T07:58:40.592512+0000","last_unstale":"2023-10-07T07:58:40.595449+0000","last_undegraded":"2023-10-07T07:58:40.595449+0000","last_fullsized":"2023-10-07T07:58:40.595449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:35:31.024079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.597077+0000","last_change":"2023-10-07T07:58:40.594194+0000","last_active":"2023-10-07T07:58:40.597077+0000","last_peered":"2023-10-07T07:58:40.597077+0000","last_clean":"2023-10-07T07:58:40.597077+0000","last_became_active":"2023-10-07T07:58:40.593929+0000","last_became_peered":"2023-10-07T07:58:40.593929+0000","last_unstale":"2023-10-07T07:58:40.597077+0000","last_undegraded":"2023-10-07T07:58:40.597077+0000","last_fullsized":"2023-10-07T07:58:40.597077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:05:32.719608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600026+0000","last_change":"2023-10-07T07:58:40.592305+0000","last_active":"2023-10-07T07:58:40.600026+0000","last_peered":"2023-10-07T07:58:40.600026+0000","last_clean":"2023-10-07T07:58:40.600026+0000","last_became_active":"2023-10-07T07:58:40.591952+0000","last_became_peered":"2023-10-07T07:58:40.591952+0000","last_unstale":"2023-10-07T07:58:40.600026+0000","last_undegraded":"2023-10-07T07:58:40.600026+0000","last_fullsized":"2023-10-07T07:58:40.600026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:39:28.581635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609304+0000","last_change":"2023-10-07T07:58:40.605681+0000","last_active":"2023-10-07T07:58:40.609304+0000","last_peered":"2023-10-07T07:58:40.609304+0000","last_clean":"2023-10-07T07:58:40.609304+0000","last_became_active":"2023-10-07T07:58:40.605394+0000","last_became_peered":"2023-10-07T07:58:40.605394+0000","last_unstale":"2023-10-07T07:58:40.609304+0000","last_undegraded":"2023-10-07T07:58:40.609304+0000","last_fullsized":"2023-10-07T07:58:40.609304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:30:21.307510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.607319+0000","last_change":"2023-10-07T07:58:40.594584+0000","last_active":"2023-10-07T07:58:40.607319+0000","last_peered":"2023-10-07T07:58:40.607319+0000","last_clean":"2023-10-07T07:58:40.607319+0000","last_became_active":"2023-10-07T07:58:40.594250+0000","last_became_peered":"2023-10-07T07:58:40.594250+0000","last_unstale":"2023-10-07T07:58:40.607319+0000","last_undegraded":"2023-10-07T07:58:40.607319+0000","last_fullsized":"2023-10-07T07:58:40.607319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:03.887160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600598+0000","last_change":"2023-10-07T07:58:40.597026+0000","last_active":"2023-10-07T07:58:40.600598+0000","last_peered":"2023-10-07T07:58:40.600598+0000","last_clean":"2023-10-07T07:58:40.600598+0000","last_became_active":"2023-10-07T07:58:40.596659+0000","last_became_peered":"2023-10-07T07:58:40.596659+0000","last_unstale":"2023-10-07T07:58:40.600598+0000","last_undegraded":"2023-10-07T07:58:40.600598+0000","last_fullsized":"2023-10-07T07:58:40.600598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:45.999293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595040+0000","last_change":"2023-10-07T07:58:40.590152+0000","last_active":"2023-10-07T07:58:40.595040+0000","last_peered":"2023-10-07T07:58:40.595040+0000","last_clean":"2023-10-07T07:58:40.595040+0000","last_became_active":"2023-10-07T07:58:40.589246+0000","last_became_peered":"2023-10-07T07:58:40.589246+0000","last_unstale":"2023-10-07T07:58:40.595040+0000","last_undegraded":"2023-10-07T07:58:40.595040+0000","last_fullsized":"2023-10-07T07:58:40.595040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:05:32.719608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609772+0000","last_change":"2023-10-07T07:58:40.589682+0000","last_active":"2023-10-07T07:58:40.609772+0000","last_peered":"2023-10-07T07:58:40.609772+0000","last_clean":"2023-10-07T07:58:40.609772+0000","last_became_active":"2023-10-07T07:58:40.589437+0000","last_became_peered":"2023-10-07T07:58:40.589437+0000","last_unstale":"2023-10-07T07:58:40.609772+0000","last_undegraded":"2023-10-07T07:58:40.609772+0000","last_fullsized":"2023-10-07T07:58:40.609772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:39:56.102863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600394+0000","last_change":"2023-10-07T07:58:40.596580+0000","last_active":"2023-10-07T07:58:40.600394+0000","last_peered":"2023-10-07T07:58:40.600394+0000","last_clean":"2023-10-07T07:58:40.600394+0000","last_became_active":"2023-10-07T07:58:40.596270+0000","last_became_peered":"2023-10-07T07:58:40.596270+0000","last_unstale":"2023-10-07T07:58:40.600394+0000","last_undegraded":"2023-10-07T07:58:40.600394+0000","last_fullsized":"2023-10-07T07:58:40.600394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:58:50.796780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.593166+0000","last_change":"2023-10-07T07:58:40.587912+0000","last_active":"2023-10-07T07:58:40.593166+0000","last_peered":"2023-10-07T07:58:40.593166+0000","last_clean":"2023-10-07T07:58:40.593166+0000","last_became_active":"2023-10-07T07:58:40.586751+0000","last_became_peered":"2023-10-07T07:58:40.586751+0000","last_unstale":"2023-10-07T07:58:40.593166+0000","last_undegraded":"2023-10-07T07:58:40.593166+0000","last_fullsized":"2023-10-07T07:58:40.593166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:32:36.691334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.594593+0000","last_change":"2023-10-07T07:58:40.591131+0000","last_active":"2023-10-07T07:58:40.594593+0000","last_peered":"2023-10-07T07:58:40.594593+0000","last_clean":"2023-10-07T07:58:40.594593+0000","last_became_active":"2023-10-07T07:58:40.588875+0000","last_became_peered":"2023-10-07T07:58:40.588875+0000","last_unstale":"2023-10-07T07:58:40.594593+0000","last_undegraded":"2023-10-07T07:58:40.594593+0000","last_fullsized":"2023-10-07T07:58:40.594593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:01.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.599445+0000","last_change":"2023-10-07T07:58:40.590081+0000","last_active":"2023-10-07T07:58:40.599445+0000","last_peered":"2023-10-07T07:58:40.599445+0000","last_clean":"2023-10-07T07:58:40.599445+0000","last_became_active":"2023-10-07T07:58:40.589665+0000","last_became_peered":"2023-10-07T07:58:40.589665+0000","last_unstale":"2023-10-07T07:58:40.599445+0000","last_undegraded":"2023-10-07T07:58:40.599445+0000","last_fullsized":"2023-10-07T07:58:40.599445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:03.887160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604397+0000","last_change":"2023-10-07T07:58:40.593644+0000","last_active":"2023-10-07T07:58:40.604397+0000","last_peered":"2023-10-07T07:58:40.604397+0000","last_clean":"2023-10-07T07:58:40.604397+0000","last_became_active":"2023-10-07T07:58:40.593328+0000","last_became_peered":"2023-10-07T07:58:40.593328+0000","last_unstale":"2023-10-07T07:58:40.604397+0000","last_undegraded":"2023-10-07T07:58:40.604397+0000","last_fullsized":"2023-10-07T07:58:40.604397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:14.250170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604962+0000","last_change":"2023-10-07T07:58:40.589158+0000","last_active":"2023-10-07T07:58:40.604962+0000","last_peered":"2023-10-07T07:58:40.604962+0000","last_clean":"2023-10-07T07:58:40.604962+0000","last_became_active":"2023-10-07T07:58:40.588866+0000","last_became_peered":"2023-10-07T07:58:40.588866+0000","last_unstale":"2023-10-07T07:58:40.604962+0000","last_undegraded":"2023-10-07T07:58:40.604962+0000","last_fullsized":"2023-10-07T07:58:40.604962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:01:14.530812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595309+0000","last_change":"2023-10-07T07:58:40.590179+0000","last_active":"2023-10-07T07:58:40.595309+0000","last_peered":"2023-10-07T07:58:40.595309+0000","last_clean":"2023-10-07T07:58:40.595309+0000","last_became_active":"2023-10-07T07:58:40.589938+0000","last_became_peered":"2023-10-07T07:58:40.589938+0000","last_unstale":"2023-10-07T07:58:40.595309+0000","last_undegraded":"2023-10-07T07:58:40.595309+0000","last_fullsized":"2023-10-07T07:58:40.595309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:31.416135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602551+0000","last_change":"2023-10-07T07:58:40.598761+0000","last_active":"2023-10-07T07:58:40.602551+0000","last_peered":"2023-10-07T07:58:40.602551+0000","last_clean":"2023-10-07T07:58:40.602551+0000","last_became_active":"2023-10-07T07:58:40.598455+0000","last_became_peered":"2023-10-07T07:58:40.598455+0000","last_unstale":"2023-10-07T07:58:40.602551+0000","last_undegraded":"2023-10-07T07:58:40.602551+0000","last_fullsized":"2023-10-07T07:58:40.602551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:45.999293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602439+0000","last_change":"2023-10-07T07:58:40.598934+0000","last_active":"2023-10-07T07:58:40.602439+0000","last_peered":"2023-10-07T07:58:40.602439+0000","last_clean":"2023-10-07T07:58:40.602439+0000","last_became_active":"2023-10-07T07:58:40.598333+0000","last_became_peered":"2023-10-07T07:58:40.598333+0000","last_unstale":"2023-10-07T07:58:40.602439+0000","last_undegraded":"2023-10-07T07:58:40.602439+0000","last_fullsized":"2023-10-07T07:58:40.602439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:30:21.307510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604266+0000","last_change":"2023-10-07T07:58:40.593825+0000","last_active":"2023-10-07T07:58:40.604266+0000","last_peered":"2023-10-07T07:58:40.604266+0000","last_clean":"2023-10-07T07:58:40.604266+0000","last_became_active":"2023-10-07T07:58:40.591700+0000","last_became_peered":"2023-10-07T07:58:40.591700+0000","last_unstale":"2023-10-07T07:58:40.604266+0000","last_undegraded":"2023-10-07T07:58:40.604266+0000","last_fullsized":"2023-10-07T07:58:40.604266+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:05:32.719608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609172+0000","last_change":"2023-10-07T07:58:40.600187+0000","last_active":"2023-10-07T07:58:40.609172+0000","last_peered":"2023-10-07T07:58:40.609172+0000","last_clean":"2023-10-07T07:58:40.609172+0000","last_became_active":"2023-10-07T07:58:40.599706+0000","last_became_peered":"2023-10-07T07:58:40.599706+0000","last_unstale":"2023-10-07T07:58:40.609172+0000","last_undegraded":"2023-10-07T07:58:40.609172+0000","last_fullsized":"2023-10-07T07:58:40.609172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:14.250170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602930+0000","last_change":"2023-10-07T07:58:40.601412+0000","last_active":"2023-10-07T07:58:40.602930+0000","last_peered":"2023-10-07T07:58:40.602930+0000","last_clean":"2023-10-07T07:58:40.602930+0000","last_became_active":"2023-10-07T07:58:40.601100+0000","last_became_peered":"2023-10-07T07:58:40.601100+0000","last_unstale":"2023-10-07T07:58:40.602930+0000","last_undegraded":"2023-10-07T07:58:40.602930+0000","last_fullsized":"2023-10-07T07:58:40.602930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:49:15.409580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609052+0000","last_change":"2023-10-07T07:58:40.599486+0000","last_active":"2023-10-07T07:58:40.609052+0000","last_peered":"2023-10-07T07:58:40.609052+0000","last_clean":"2023-10-07T07:58:40.609052+0000","last_became_active":"2023-10-07T07:58:40.599227+0000","last_became_peered":"2023-10-07T07:58:40.599227+0000","last_unstale":"2023-10-07T07:58:40.609052+0000","last_undegraded":"2023-10-07T07:58:40.609052+0000","last_fullsized":"2023-10-07T07:58:40.609052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:05:32.719608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.603000+0000","last_change":"2023-10-07T07:58:40.596634+0000","last_active":"2023-10-07T07:58:40.603000+0000","last_peered":"2023-10-07T07:58:40.603000+0000","last_clean":"2023-10-07T07:58:40.603000+0000","last_became_active":"2023-10-07T07:58:40.596013+0000","last_became_peered":"2023-10-07T07:58:40.596013+0000","last_unstale":"2023-10-07T07:58:40.603000+0000","last_undegraded":"2023-10-07T07:58:40.603000+0000","last_fullsized":"2023-10-07T07:58:40.603000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:02:38.932523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604298+0000","last_change":"2023-10-07T07:58:40.601340+0000","last_active":"2023-10-07T07:58:40.604298+0000","last_peered":"2023-10-07T07:58:40.604298+0000","last_clean":"2023-10-07T07:58:40.604298+0000","last_became_active":"2023-10-07T07:58:40.601090+0000","last_became_peered":"2023-10-07T07:58:40.601090+0000","last_unstale":"2023-10-07T07:58:40.604298+0000","last_undegraded":"2023-10-07T07:58:40.604298+0000","last_fullsized":"2023-10-07T07:58:40.604298+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:04:39.135155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.605008+0000","last_change":"2023-10-07T07:58:40.604076+0000","last_active":"2023-10-07T07:58:40.605008+0000","last_peered":"2023-10-07T07:58:40.605008+0000","last_clean":"2023-10-07T07:58:40.605008+0000","last_became_active":"2023-10-07T07:58:40.603798+0000","last_became_peered":"2023-10-07T07:58:40.603798+0000","last_unstale":"2023-10-07T07:58:40.605008+0000","last_undegraded":"2023-10-07T07:58:40.605008+0000","last_fullsized":"2023-10-07T07:58:40.605008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:04:39.135155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.607476+0000","last_change":"2023-10-07T07:58:40.594699+0000","last_active":"2023-10-07T07:58:40.607476+0000","last_peered":"2023-10-07T07:58:40.607476+0000","last_clean":"2023-10-07T07:58:40.607476+0000","last_became_active":"2023-10-07T07:58:40.594300+0000","last_became_peered":"2023-10-07T07:58:40.594300+0000","last_unstale":"2023-10-07T07:58:40.607476+0000","last_undegraded":"2023-10-07T07:58:40.607476+0000","last_fullsized":"2023-10-07T07:58:40.607476+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:01.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604288+0000","last_change":"2023-10-07T07:58:40.596980+0000","last_active":"2023-10-07T07:58:40.604288+0000","last_peered":"2023-10-07T07:58:40.604288+0000","last_clean":"2023-10-07T07:58:40.604288+0000","last_became_active":"2023-10-07T07:58:40.596706+0000","last_became_peered":"2023-10-07T07:58:40.596706+0000","last_unstale":"2023-10-07T07:58:40.604288+0000","last_undegraded":"2023-10-07T07:58:40.604288+0000","last_fullsized":"2023-10-07T07:58:40.604288+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:01.666317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604178+0000","last_change":"2023-10-07T07:58:40.598661+0000","last_active":"2023-10-07T07:58:40.604178+0000","last_peered":"2023-10-07T07:58:40.604178+0000","last_clean":"2023-10-07T07:58:40.604178+0000","last_became_active":"2023-10-07T07:58:40.598334+0000","last_became_peered":"2023-10-07T07:58:40.598334+0000","last_unstale":"2023-10-07T07:58:40.604178+0000","last_undegraded":"2023-10-07T07:58:40.604178+0000","last_fullsized":"2023-10-07T07:58:40.604178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:04:39.135155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609284+0000","last_change":"2023-10-07T07:58:40.605521+0000","last_active":"2023-10-07T07:58:40.609284+0000","last_peered":"2023-10-07T07:58:40.609284+0000","last_clean":"2023-10-07T07:58:40.609284+0000","last_became_active":"2023-10-07T07:58:40.603261+0000","last_became_peered":"2023-10-07T07:58:40.603261+0000","last_unstale":"2023-10-07T07:58:40.609284+0000","last_undegraded":"2023-10-07T07:58:40.609284+0000","last_fullsized":"2023-10-07T07:58:40.609284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:59:15.735667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.592959+0000","last_change":"2023-10-07T07:58:40.585021+0000","last_active":"2023-10-07T07:58:40.592959+0000","last_peered":"2023-10-07T07:58:40.592959+0000","last_clean":"2023-10-07T07:58:40.592959+0000","last_became_active":"2023-10-07T07:58:40.584726+0000","last_became_peered":"2023-10-07T07:58:40.584726+0000","last_unstale":"2023-10-07T07:58:40.592959+0000","last_undegraded":"2023-10-07T07:58:40.592959+0000","last_fullsized":"2023-10-07T07:58:40.592959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:51:17.036352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.580215+0000","last_change":"2023-10-07T07:58:40.578940+0000","last_active":"2023-10-07T07:58:40.580215+0000","last_peered":"2023-10-07T07:58:40.580215+0000","last_clean":"2023-10-07T07:58:40.580215+0000","last_became_active":"2023-10-07T07:58:40.578651+0000","last_became_peered":"2023-10-07T07:58:40.578651+0000","last_unstale":"2023-10-07T07:58:40.580215+0000","last_undegraded":"2023-10-07T07:58:40.580215+0000","last_fullsized":"2023-10-07T07:58:40.580215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:02:58.823561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.611972+0000","last_change":"2023-10-07T07:58:39.587306+0000","last_active":"2023-10-07T07:58:40.611972+0000","last_peered":"2023-10-07T07:58:40.611972+0000","last_clean":"2023-10-07T07:58:40.611972+0000","last_became_active":"2023-10-07T07:58:39.587000+0000","last_became_peered":"2023-10-07T07:58:39.587000+0000","last_unstale":"2023-10-07T07:58:40.611972+0000","last_undegraded":"2023-10-07T07:58:40.611972+0000","last_fullsized":"2023-10-07T07:58:40.611972+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:02:37.926625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.605736+0000","last_change":"2023-10-07T07:58:40.593542+0000","last_active":"2023-10-07T07:58:40.605736+0000","last_peered":"2023-10-07T07:58:40.605736+0000","last_clean":"2023-10-07T07:58:40.605736+0000","last_became_active":"2023-10-07T07:58:40.589440+0000","last_became_peered":"2023-10-07T07:58:40.589440+0000","last_unstale":"2023-10-07T07:58:40.605736+0000","last_undegraded":"2023-10-07T07:58:40.605736+0000","last_fullsized":"2023-10-07T07:58:40.605736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:36:03.367218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609462+0000","last_change":"2023-10-07T07:58:39.583198+0000","last_active":"2023-10-07T07:58:40.609462+0000","last_peered":"2023-10-07T07:58:40.609462+0000","last_clean":"2023-10-07T07:58:40.609462+0000","last_became_active":"2023-10-07T07:58:39.582404+0000","last_became_peered":"2023-10-07T07:58:39.582404+0000","last_unstale":"2023-10-07T07:58:40.609462+0000","last_undegraded":"2023-10-07T07:58:40.609462+0000","last_fullsized":"2023-10-07T07:58:40.609462+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:58:49.790881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609009+0000","last_change":"2023-10-07T07:58:40.599242+0000","last_active":"2023-10-07T07:58:40.609009+0000","last_peered":"2023-10-07T07:58:40.609009+0000","last_clean":"2023-10-07T07:58:40.609009+0000","last_became_active":"2023-10-07T07:58:40.598952+0000","last_became_peered":"2023-10-07T07:58:40.598952+0000","last_unstale":"2023-10-07T07:58:40.609009+0000","last_undegraded":"2023-10-07T07:58:40.609009+0000","last_fullsized":"2023-10-07T07:58:40.609009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:04:39.135155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.603292+0000","last_change":"2023-10-07T07:58:39.585946+0000","last_active":"2023-10-07T07:58:40.603292+0000","last_peered":"2023-10-07T07:58:40.603292+0000","last_clean":"2023-10-07T07:58:40.603292+0000","last_became_active":"2023-10-07T07:58:39.585658+0000","last_became_peered":"2023-10-07T07:58:39.585658+0000","last_unstale":"2023-10-07T07:58:40.603292+0000","last_undegraded":"2023-10-07T07:58:40.603292+0000","last_fullsized":"2023-10-07T07:58:40.603292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:09:25.972791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.590126+0000","last_change":"2023-10-07T07:58:40.588737+0000","last_active":"2023-10-07T07:58:40.590126+0000","last_peered":"2023-10-07T07:58:40.590126+0000","last_clean":"2023-10-07T07:58:40.590126+0000","last_became_active":"2023-10-07T07:58:40.588160+0000","last_became_peered":"2023-10-07T07:58:40.588160+0000","last_unstale":"2023-10-07T07:58:40.590126+0000","last_undegraded":"2023-10-07T07:58:40.590126+0000","last_fullsized":"2023-10-07T07:58:40.590126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:03.887160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604097+0000","last_change":"2023-10-07T07:58:39.590216+0000","last_active":"2023-10-07T07:58:40.604097+0000","last_peered":"2023-10-07T07:58:40.604097+0000","last_clean":"2023-10-07T07:58:40.604097+0000","last_became_active":"2023-10-07T07:58:39.589842+0000","last_became_peered":"2023-10-07T07:58:39.589842+0000","last_unstale":"2023-10-07T07:58:40.604097+0000","last_undegraded":"2023-10-07T07:58:40.604097+0000","last_fullsized":"2023-10-07T07:58:40.604097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:00:59.089825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.597974+0000","last_change":"2023-10-07T07:58:40.596239+0000","last_active":"2023-10-07T07:58:40.597974+0000","last_peered":"2023-10-07T07:58:40.597974+0000","last_clean":"2023-10-07T07:58:40.597974+0000","last_became_active":"2023-10-07T07:58:40.595905+0000","last_became_peered":"2023-10-07T07:58:40.595905+0000","last_unstale":"2023-10-07T07:58:40.597974+0000","last_undegraded":"2023-10-07T07:58:40.597974+0000","last_fullsized":"2023-10-07T07:58:40.597974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:31.416135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.608691+0000","last_change":"2023-10-07T07:58:39.592109+0000","last_active":"2023-10-07T07:58:40.608691+0000","last_peered":"2023-10-07T07:58:40.608691+0000","last_clean":"2023-10-07T07:58:40.608691+0000","last_became_active":"2023-10-07T07:58:39.591824+0000","last_became_peered":"2023-10-07T07:58:39.591824+0000","last_unstale":"2023-10-07T07:58:40.608691+0000","last_undegraded":"2023-10-07T07:58:40.608691+0000","last_fullsized":"2023-10-07T07:58:40.608691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:02:50.626593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609232+0000","last_change":"2023-10-07T07:58:40.604850+0000","last_active":"2023-10-07T07:58:40.609232+0000","last_peered":"2023-10-07T07:58:40.609232+0000","last_clean":"2023-10-07T07:58:40.609232+0000","last_became_active":"2023-10-07T07:58:40.600203+0000","last_became_peered":"2023-10-07T07:58:40.600203+0000","last_unstale":"2023-10-07T07:58:40.609232+0000","last_undegraded":"2023-10-07T07:58:40.609232+0000","last_fullsized":"2023-10-07T07:58:40.609232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:46:05.776616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.608791+0000","last_change":"2023-10-07T07:58:39.594140+0000","last_active":"2023-10-07T07:58:40.608791+0000","last_peered":"2023-10-07T07:58:40.608791+0000","last_clean":"2023-10-07T07:58:40.608791+0000","last_became_active":"2023-10-07T07:58:39.593773+0000","last_became_peered":"2023-10-07T07:58:39.593773+0000","last_unstale":"2023-10-07T07:58:40.608791+0000","last_undegraded":"2023-10-07T07:58:40.608791+0000","last_fullsized":"2023-10-07T07:58:40.608791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:02:50.444828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.610055+0000","last_change":"2023-10-07T07:58:40.605444+0000","last_active":"2023-10-07T07:58:40.610055+0000","last_peered":"2023-10-07T07:58:40.610055+0000","last_clean":"2023-10-07T07:58:40.610055+0000","last_became_active":"2023-10-07T07:58:40.605135+0000","last_became_peered":"2023-10-07T07:58:40.605135+0000","last_unstale":"2023-10-07T07:58:40.610055+0000","last_undegraded":"2023-10-07T07:58:40.610055+0000","last_fullsized":"2023-10-07T07:58:40.610055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:02:51.450726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.582616+0000","last_change":"2023-10-07T07:58:39.585602+0000","last_active":"2023-10-07T07:58:40.582616+0000","last_peered":"2023-10-07T07:58:40.582616+0000","last_clean":"2023-10-07T07:58:40.582616+0000","last_became_active":"2023-10-07T07:58:39.585291+0000","last_became_peered":"2023-10-07T07:58:39.585291+0000","last_unstale":"2023-10-07T07:58:40.582616+0000","last_undegraded":"2023-10-07T07:58:40.582616+0000","last_fullsized":"2023-10-07T07:58:40.582616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:03.308648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595444+0000","last_change":"2023-10-07T07:58:40.593699+0000","last_active":"2023-10-07T07:58:40.595444+0000","last_peered":"2023-10-07T07:58:40.595444+0000","last_clean":"2023-10-07T07:58:40.595444+0000","last_became_active":"2023-10-07T07:58:40.593452+0000","last_became_peered":"2023-10-07T07:58:40.593452+0000","last_unstale":"2023-10-07T07:58:40.595444+0000","last_undegraded":"2023-10-07T07:58:40.595444+0000","last_fullsized":"2023-10-07T07:58:40.595444+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:04:36.620305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.581824+0000","last_change":"2023-10-07T07:58:39.600505+0000","last_active":"2023-10-07T07:58:40.581824+0000","last_peered":"2023-10-07T07:58:40.581824+0000","last_clean":"2023-10-07T07:58:40.581824+0000","last_became_active":"2023-10-07T07:58:39.600091+0000","last_became_peered":"2023-10-07T07:58:39.600091+0000","last_unstale":"2023-10-07T07:58:40.581824+0000","last_undegraded":"2023-10-07T07:58:40.581824+0000","last_fullsized":"2023-10-07T07:58:40.581824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:03:13.602007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.592599+0000","last_change":"2023-10-07T07:58:40.582833+0000","last_active":"2023-10-07T07:58:40.592599+0000","last_peered":"2023-10-07T07:58:40.592599+0000","last_clean":"2023-10-07T07:58:40.592599+0000","last_became_active":"2023-10-07T07:58:40.582551+0000","last_became_peered":"2023-10-07T07:58:40.582551+0000","last_unstale":"2023-10-07T07:58:40.592599+0000","last_undegraded":"2023-10-07T07:58:40.592599+0000","last_fullsized":"2023-10-07T07:58:40.592599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:10:38.683774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.585774+0000","last_change":"2023-10-07T07:58:39.587730+0000","last_active":"2023-10-07T07:58:40.585774+0000","last_peered":"2023-10-07T07:58:40.585774+0000","last_clean":"2023-10-07T07:58:40.585774+0000","last_became_active":"2023-10-07T07:58:39.587461+0000","last_became_peered":"2023-10-07T07:58:39.587461+0000","last_unstale":"2023-10-07T07:58:40.585774+0000","last_undegraded":"2023-10-07T07:58:40.585774+0000","last_fullsized":"2023-10-07T07:58:40.585774+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:36:02.361320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.592212+0000","last_change":"2023-10-07T07:58:40.589841+0000","last_active":"2023-10-07T07:58:40.592212+0000","last_peered":"2023-10-07T07:58:40.592212+0000","last_clean":"2023-10-07T07:58:40.592212+0000","last_became_active":"2023-10-07T07:58:40.589503+0000","last_became_peered":"2023-10-07T07:58:40.589503+0000","last_unstale":"2023-10-07T07:58:40.592212+0000","last_undegraded":"2023-10-07T07:58:40.592212+0000","last_fullsized":"2023-10-07T07:58:40.592212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:31:14.816056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.585727+0000","last_change":"2023-10-07T07:58:39.586723+0000","last_active":"2023-10-07T07:58:40.585727+0000","last_peered":"2023-10-07T07:58:40.585727+0000","last_clean":"2023-10-07T07:58:40.585727+0000","last_became_active":"2023-10-07T07:58:39.586265+0000","last_became_peered":"2023-10-07T07:58:39.586265+0000","last_unstale":"2023-10-07T07:58:40.585727+0000","last_undegraded":"2023-10-07T07:58:40.585727+0000","last_fullsized":"2023-10-07T07:58:40.585727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:08:19.384290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595467+0000","last_change":"2023-10-07T07:58:40.592828+0000","last_active":"2023-10-07T07:58:40.595467+0000","last_peered":"2023-10-07T07:58:40.595467+0000","last_clean":"2023-10-07T07:58:40.595467+0000","last_became_active":"2023-10-07T07:58:40.590497+0000","last_became_peered":"2023-10-07T07:58:40.590497+0000","last_unstale":"2023-10-07T07:58:40.595467+0000","last_undegraded":"2023-10-07T07:58:40.595467+0000","last_fullsized":"2023-10-07T07:58:40.595467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:06:51.002473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.611831+0000","last_change":"2023-10-07T07:58:39.591656+0000","last_active":"2023-10-07T07:58:40.611831+0000","last_peered":"2023-10-07T07:58:40.611831+0000","last_clean":"2023-10-07T07:58:40.611831+0000","last_became_active":"2023-10-07T07:58:39.588856+0000","last_became_peered":"2023-10-07T07:58:39.588856+0000","last_unstale":"2023-10-07T07:58:40.611831+0000","last_undegraded":"2023-10-07T07:58:40.611831+0000","last_fullsized":"2023-10-07T07:58:40.611831+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:31:13.810157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609792+0000","last_change":"2023-10-07T07:58:40.589887+0000","last_active":"2023-10-07T07:58:40.609792+0000","last_peered":"2023-10-07T07:58:40.609792+0000","last_clean":"2023-10-07T07:58:40.609792+0000","last_became_active":"2023-10-07T07:58:40.589590+0000","last_became_peered":"2023-10-07T07:58:40.589590+0000","last_unstale":"2023-10-07T07:58:40.609792+0000","last_undegraded":"2023-10-07T07:58:40.609792+0000","last_fullsized":"2023-10-07T07:58:40.609792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:17:19.907560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.610739+0000","last_change":"2023-10-07T07:58:39.583848+0000","last_active":"2023-10-07T07:58:40.610739+0000","last_peered":"2023-10-07T07:58:40.610739+0000","last_clean":"2023-10-07T07:58:40.610739+0000","last_became_active":"2023-10-07T07:58:39.583137+0000","last_became_peered":"2023-10-07T07:58:39.583137+0000","last_unstale":"2023-10-07T07:58:40.610739+0000","last_undegraded":"2023-10-07T07:58:40.610739+0000","last_fullsized":"2023-10-07T07:58:40.610739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:30.410237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602551+0000","last_change":"2023-10-07T07:58:40.597566+0000","last_active":"2023-10-07T07:58:40.602551+0000","last_peered":"2023-10-07T07:58:40.602551+0000","last_clean":"2023-10-07T07:58:40.602551+0000","last_became_active":"2023-10-07T07:58:40.597266+0000","last_became_peered":"2023-10-07T07:58:40.597266+0000","last_unstale":"2023-10-07T07:58:40.602551+0000","last_undegraded":"2023-10-07T07:58:40.602551+0000","last_fullsized":"2023-10-07T07:58:40.602551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:29:17.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609173+0000","last_change":"2023-10-07T07:58:39.593903+0000","last_active":"2023-10-07T07:58:40.609173+0000","last_peered":"2023-10-07T07:58:40.609173+0000","last_clean":"2023-10-07T07:58:40.609173+0000","last_became_active":"2023-10-07T07:58:39.593607+0000","last_became_peered":"2023-10-07T07:58:39.593607+0000","last_unstale":"2023-10-07T07:58:40.609173+0000","last_undegraded":"2023-10-07T07:58:40.609173+0000","last_fullsized":"2023-10-07T07:58:40.609173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:56:27.881180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.590170+0000","last_change":"2023-10-07T07:58:40.588847+0000","last_active":"2023-10-07T07:58:40.590170+0000","last_peered":"2023-10-07T07:58:40.590170+0000","last_clean":"2023-10-07T07:58:40.590170+0000","last_became_active":"2023-10-07T07:58:40.588336+0000","last_became_peered":"2023-10-07T07:58:40.588336+0000","last_unstale":"2023-10-07T07:58:40.590170+0000","last_undegraded":"2023-10-07T07:58:40.590170+0000","last_fullsized":"2023-10-07T07:58:40.590170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:21:39.170465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.580148+0000","last_change":"2023-10-07T07:58:39.584100+0000","last_active":"2023-10-07T07:58:40.580148+0000","last_peered":"2023-10-07T07:58:40.580148+0000","last_clean":"2023-10-07T07:58:40.580148+0000","last_became_active":"2023-10-07T07:58:39.583747+0000","last_became_peered":"2023-10-07T07:58:39.583747+0000","last_unstale":"2023-10-07T07:58:40.580148+0000","last_undegraded":"2023-10-07T07:58:40.580148+0000","last_fullsized":"2023-10-07T07:58:40.580148+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:44.993394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.609294+0000","last_change":"2023-10-07T07:58:40.603617+0000","last_active":"2023-10-07T07:58:40.609294+0000","last_peered":"2023-10-07T07:58:40.609294+0000","last_clean":"2023-10-07T07:58:40.609294+0000","last_became_active":"2023-10-07T07:58:40.599942+0000","last_became_peered":"2023-10-07T07:58:40.599942+0000","last_unstale":"2023-10-07T07:58:40.609294+0000","last_undegraded":"2023-10-07T07:58:40.609294+0000","last_fullsized":"2023-10-07T07:58:40.609294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:29:17.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.578841+0000","last_change":"2023-10-07T07:58:39.586423+0000","last_active":"2023-10-07T07:58:40.578841+0000","last_peered":"2023-10-07T07:58:40.578841+0000","last_clean":"2023-10-07T07:58:40.578841+0000","last_became_active":"2023-10-07T07:58:39.586121+0000","last_became_peered":"2023-10-07T07:58:39.586121+0000","last_unstale":"2023-10-07T07:58:40.578841+0000","last_undegraded":"2023-10-07T07:58:40.578841+0000","last_fullsized":"2023-10-07T07:58:40.578841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:39:27.575737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.605801+0000","last_change":"2023-10-07T07:58:40.588568+0000","last_active":"2023-10-07T07:58:40.605801+0000","last_peered":"2023-10-07T07:58:40.605801+0000","last_clean":"2023-10-07T07:58:40.605801+0000","last_became_active":"2023-10-07T07:58:40.587823+0000","last_became_peered":"2023-10-07T07:58:40.587823+0000","last_unstale":"2023-10-07T07:58:40.605801+0000","last_undegraded":"2023-10-07T07:58:40.605801+0000","last_fullsized":"2023-10-07T07:58:40.605801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:35:36.270143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.611870+0000","last_change":"2023-10-07T07:58:39.585927+0000","last_active":"2023-10-07T07:58:40.611870+0000","last_peered":"2023-10-07T07:58:40.611870+0000","last_clean":"2023-10-07T07:58:40.611870+0000","last_became_active":"2023-10-07T07:58:39.585622+0000","last_became_peered":"2023-10-07T07:58:39.585622+0000","last_unstale":"2023-10-07T07:58:40.611870+0000","last_undegraded":"2023-10-07T07:58:40.611870+0000","last_fullsized":"2023-10-07T07:58:40.611870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:26:31.499283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.603981+0000","last_change":"2023-10-07T07:58:40.588526+0000","last_active":"2023-10-07T07:58:40.603981+0000","last_peered":"2023-10-07T07:58:40.603981+0000","last_clean":"2023-10-07T07:58:40.603981+0000","last_became_active":"2023-10-07T07:58:40.588208+0000","last_became_peered":"2023-10-07T07:58:40.588208+0000","last_unstale":"2023-10-07T07:58:40.603981+0000","last_undegraded":"2023-10-07T07:58:40.603981+0000","last_fullsized":"2023-10-07T07:58:40.603981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:25:55.442476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.603237+0000","last_change":"2023-10-07T07:58:39.588571+0000","last_active":"2023-10-07T07:58:40.603237+0000","last_peered":"2023-10-07T07:58:40.603237+0000","last_clean":"2023-10-07T07:58:40.603237+0000","last_became_active":"2023-10-07T07:58:39.588297+0000","last_became_peered":"2023-10-07T07:58:39.588297+0000","last_unstale":"2023-10-07T07:58:40.603237+0000","last_undegraded":"2023-10-07T07:58:40.603237+0000","last_fullsized":"2023-10-07T07:58:40.603237+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:34:00.214894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.610120+0000","last_change":"2023-10-07T07:58:40.606422+0000","last_active":"2023-10-07T07:58:40.610120+0000","last_peered":"2023-10-07T07:58:40.610120+0000","last_clean":"2023-10-07T07:58:40.610120+0000","last_became_active":"2023-10-07T07:58:40.605829+0000","last_became_peered":"2023-10-07T07:58:40.605829+0000","last_unstale":"2023-10-07T07:58:40.610120+0000","last_undegraded":"2023-10-07T07:58:40.610120+0000","last_fullsized":"2023-10-07T07:58:40.610120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:10:32.121446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.608766+0000","last_change":"2023-10-07T07:58:39.594216+0000","last_active":"2023-10-07T07:58:40.608766+0000","last_peered":"2023-10-07T07:58:40.608766+0000","last_clean":"2023-10-07T07:58:40.608766+0000","last_became_active":"2023-10-07T07:58:39.593897+0000","last_became_peered":"2023-10-07T07:58:39.593897+0000","last_unstale":"2023-10-07T07:58:40.608766+0000","last_undegraded":"2023-10-07T07:58:40.608766+0000","last_fullsized":"2023-10-07T07:58:40.608766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:10:31.115548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602822+0000","last_change":"2023-10-07T07:58:40.598706+0000","last_active":"2023-10-07T07:58:40.602822+0000","last_peered":"2023-10-07T07:58:40.602822+0000","last_clean":"2023-10-07T07:58:40.602822+0000","last_became_active":"2023-10-07T07:58:40.598438+0000","last_became_peered":"2023-10-07T07:58:40.598438+0000","last_unstale":"2023-10-07T07:58:40.602822+0000","last_undegraded":"2023-10-07T07:58:40.602822+0000","last_fullsized":"2023-10-07T07:58:40.602822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:03.887160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.611347+0000","last_change":"2023-10-07T07:58:39.591836+0000","last_active":"2023-10-07T07:58:40.611347+0000","last_peered":"2023-10-07T07:58:40.611347+0000","last_clean":"2023-10-07T07:58:40.611347+0000","last_became_active":"2023-10-07T07:58:39.590987+0000","last_became_peered":"2023-10-07T07:58:39.590987+0000","last_unstale":"2023-10-07T07:58:40.611347+0000","last_undegraded":"2023-10-07T07:58:40.611347+0000","last_fullsized":"2023-10-07T07:58:40.611347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:04:35.614406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602856+0000","last_change":"2023-10-07T07:58:40.599435+0000","last_active":"2023-10-07T07:58:40.602856+0000","last_peered":"2023-10-07T07:58:40.602856+0000","last_clean":"2023-10-07T07:58:40.602856+0000","last_became_active":"2023-10-07T07:58:40.599166+0000","last_became_peered":"2023-10-07T07:58:40.599166+0000","last_unstale":"2023-10-07T07:58:40.602856+0000","last_undegraded":"2023-10-07T07:58:40.602856+0000","last_fullsized":"2023-10-07T07:58:40.602856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:01:00.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.585520+0000","last_change":"2023-10-07T07:58:39.590387+0000","last_active":"2023-10-07T07:58:40.585520+0000","last_peered":"2023-10-07T07:58:40.585520+0000","last_clean":"2023-10-07T07:58:40.585520+0000","last_became_active":"2023-10-07T07:58:39.590051+0000","last_became_peered":"2023-10-07T07:58:39.590051+0000","last_unstale":"2023-10-07T07:58:40.585520+0000","last_undegraded":"2023-10-07T07:58:40.585520+0000","last_fullsized":"2023-10-07T07:58:40.585520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:10:36.901386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.596736+0000","last_change":"2023-10-07T07:58:40.591574+0000","last_active":"2023-10-07T07:58:40.596736+0000","last_peered":"2023-10-07T07:58:40.596736+0000","last_clean":"2023-10-07T07:58:40.596736+0000","last_became_active":"2023-10-07T07:58:40.591330+0000","last_became_peered":"2023-10-07T07:58:40.591330+0000","last_unstale":"2023-10-07T07:58:40.596736+0000","last_undegraded":"2023-10-07T07:58:40.596736+0000","last_fullsized":"2023-10-07T07:58:40.596736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:59:15.735667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604621+0000","last_change":"2023-10-07T07:58:39.590861+0000","last_active":"2023-10-07T07:58:40.604621+0000","last_peered":"2023-10-07T07:58:40.604621+0000","last_clean":"2023-10-07T07:58:40.604621+0000","last_became_active":"2023-10-07T07:58:39.590587+0000","last_became_peered":"2023-10-07T07:58:39.590587+0000","last_unstale":"2023-10-07T07:58:40.604621+0000","last_undegraded":"2023-10-07T07:58:40.604621+0000","last_fullsized":"2023-10-07T07:58:40.604621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:34:15.142970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.597099+0000","last_change":"2023-10-07T07:58:40.594675+0000","last_active":"2023-10-07T07:58:40.597099+0000","last_peered":"2023-10-07T07:58:40.597099+0000","last_clean":"2023-10-07T07:58:40.597099+0000","last_became_active":"2023-10-07T07:58:40.592821+0000","last_became_peered":"2023-10-07T07:58:40.592821+0000","last_unstale":"2023-10-07T07:58:40.597099+0000","last_undegraded":"2023-10-07T07:58:40.597099+0000","last_fullsized":"2023-10-07T07:58:40.597099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:14.250170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.585573+0000","last_change":"2023-10-07T07:58:39.594744+0000","last_active":"2023-10-07T07:58:40.585573+0000","last_peered":"2023-10-07T07:58:40.585573+0000","last_clean":"2023-10-07T07:58:40.585573+0000","last_became_active":"2023-10-07T07:58:39.594470+0000","last_became_peered":"2023-10-07T07:58:39.594470+0000","last_unstale":"2023-10-07T07:58:40.585573+0000","last_undegraded":"2023-10-07T07:58:40.585573+0000","last_fullsized":"2023-10-07T07:58:40.585573+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:00:59.089825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.590635+0000","last_change":"2023-10-07T07:58:40.587272+0000","last_active":"2023-10-07T07:58:40.590635+0000","last_peered":"2023-10-07T07:58:40.590635+0000","last_clean":"2023-10-07T07:58:40.590635+0000","last_became_active":"2023-10-07T07:58:40.586960+0000","last_became_peered":"2023-10-07T07:58:40.586960+0000","last_unstale":"2023-10-07T07:58:40.590635+0000","last_undegraded":"2023-10-07T07:58:40.590635+0000","last_fullsized":"2023-10-07T07:58:40.590635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:08:20.390188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.587107+0000","last_change":"2023-10-07T07:58:39.585889+0000","last_active":"2023-10-07T07:58:40.587107+0000","last_peered":"2023-10-07T07:58:40.587107+0000","last_clean":"2023-10-07T07:58:40.587107+0000","last_became_active":"2023-10-07T07:58:39.585598+0000","last_became_peered":"2023-10-07T07:58:39.585598+0000","last_unstale":"2023-10-07T07:58:40.587107+0000","last_undegraded":"2023-10-07T07:58:40.587107+0000","last_fullsized":"2023-10-07T07:58:40.587107+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:04:55.540544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.610160+0000","last_change":"2023-10-07T07:58:40.607004+0000","last_active":"2023-10-07T07:58:40.610160+0000","last_peered":"2023-10-07T07:58:40.610160+0000","last_clean":"2023-10-07T07:58:40.610160+0000","last_became_active":"2023-10-07T07:58:40.606412+0000","last_became_peered":"2023-10-07T07:58:40.606412+0000","last_unstale":"2023-10-07T07:58:40.610160+0000","last_undegraded":"2023-10-07T07:58:40.610160+0000","last_fullsized":"2023-10-07T07:58:40.610160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:01:00.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.576238+0000","last_change":"2023-10-07T07:58:39.582581+0000","last_active":"2023-10-07T07:58:40.576238+0000","last_peered":"2023-10-07T07:58:40.576238+0000","last_clean":"2023-10-07T07:58:40.576238+0000","last_became_active":"2023-10-07T07:58:39.582271+0000","last_became_peered":"2023-10-07T07:58:39.582271+0000","last_unstale":"2023-10-07T07:58:40.576238+0000","last_undegraded":"2023-10-07T07:58:40.576238+0000","last_fullsized":"2023-10-07T07:58:40.576238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:58:02.881262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.602970+0000","last_change":"2023-10-07T07:58:40.599707+0000","last_active":"2023-10-07T07:58:40.602970+0000","last_peered":"2023-10-07T07:58:40.602970+0000","last_clean":"2023-10-07T07:58:40.602970+0000","last_became_active":"2023-10-07T07:58:40.598990+0000","last_became_peered":"2023-10-07T07:58:40.598990+0000","last_unstale":"2023-10-07T07:58:40.602970+0000","last_undegraded":"2023-10-07T07:58:40.602970+0000","last_fullsized":"2023-10-07T07:58:40.602970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:02:51.450726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.607930+0000","last_change":"2023-10-07T07:58:39.584540+0000","last_active":"2023-10-07T07:58:40.607930+0000","last_peered":"2023-10-07T07:58:40.607930+0000","last_clean":"2023-10-07T07:58:40.607930+0000","last_became_active":"2023-10-07T07:58:39.584195+0000","last_became_peered":"2023-10-07T07:58:39.584195+0000","last_unstale":"2023-10-07T07:58:40.607930+0000","last_undegraded":"2023-10-07T07:58:40.607930+0000","last_fullsized":"2023-10-07T07:58:40.607930+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:00:59.089825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604983+0000","last_change":"2023-10-07T07:58:40.592731+0000","last_active":"2023-10-07T07:58:40.604983+0000","last_peered":"2023-10-07T07:58:40.604983+0000","last_clean":"2023-10-07T07:58:40.604983+0000","last_became_active":"2023-10-07T07:58:40.592468+0000","last_became_peered":"2023-10-07T07:58:40.592468+0000","last_unstale":"2023-10-07T07:58:40.604983+0000","last_undegraded":"2023-10-07T07:58:40.604983+0000","last_fullsized":"2023-10-07T07:58:40.604983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:08:24.940558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-07T07:58:41.719772+0000","last_change":"2023-10-07T07:58:39.586541+0000","last_active":"2023-10-07T07:58:41.719772+0000","last_peered":"2023-10-07T07:58:41.719772+0000","last_clean":"2023-10-07T07:58:41.719772+0000","last_became_active":"2023-10-07T07:58:39.585995+0000","last_became_peered":"2023-10-07T07:58:39.585995+0000","last_unstale":"2023-10-07T07:58:41.719772+0000","last_undegraded":"2023-10-07T07:58:41.719772+0000","last_fullsized":"2023-10-07T07:58:41.719772+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:46:04.770718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.600053+0000","last_change":"2023-10-07T07:58:40.592532+0000","last_active":"2023-10-07T07:58:40.600053+0000","last_peered":"2023-10-07T07:58:40.600053+0000","last_clean":"2023-10-07T07:58:40.600053+0000","last_became_active":"2023-10-07T07:58:40.592198+0000","last_became_peered":"2023-10-07T07:58:40.592198+0000","last_unstale":"2023-10-07T07:58:40.600053+0000","last_undegraded":"2023-10-07T07:58:40.600053+0000","last_fullsized":"2023-10-07T07:58:40.600053+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:58:50.796780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.577431+0000","last_change":"2023-10-07T07:58:39.587210+0000","last_active":"2023-10-07T07:58:40.577431+0000","last_peered":"2023-10-07T07:58:40.577431+0000","last_clean":"2023-10-07T07:58:40.577431+0000","last_became_active":"2023-10-07T07:58:39.586827+0000","last_became_peered":"2023-10-07T07:58:39.586827+0000","last_unstale":"2023-10-07T07:58:40.577431+0000","last_undegraded":"2023-10-07T07:58:40.577431+0000","last_fullsized":"2023-10-07T07:58:40.577431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:52:44.993394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604080+0000","last_change":"2023-10-07T07:58:40.593651+0000","last_active":"2023-10-07T07:58:40.604080+0000","last_peered":"2023-10-07T07:58:40.604080+0000","last_clean":"2023-10-07T07:58:40.604080+0000","last_became_active":"2023-10-07T07:58:40.593377+0000","last_became_peered":"2023-10-07T07:58:40.593377+0000","last_unstale":"2023-10-07T07:58:40.604080+0000","last_undegraded":"2023-10-07T07:58:40.604080+0000","last_fullsized":"2023-10-07T07:58:40.604080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:59:15.735667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.576907+0000","last_change":"2023-10-07T07:58:39.584604+0000","last_active":"2023-10-07T07:58:40.576907+0000","last_peered":"2023-10-07T07:58:40.576907+0000","last_clean":"2023-10-07T07:58:40.576907+0000","last_became_active":"2023-10-07T07:58:39.584209+0000","last_became_peered":"2023-10-07T07:58:39.584209+0000","last_unstale":"2023-10-07T07:58:40.576907+0000","last_undegraded":"2023-10-07T07:58:40.576907+0000","last_fullsized":"2023-10-07T07:58:40.576907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:10:31.115548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.595075+0000","last_change":"2023-10-07T07:58:40.592093+0000","last_active":"2023-10-07T07:58:40.595075+0000","last_peered":"2023-10-07T07:58:40.595075+0000","last_clean":"2023-10-07T07:58:40.595075+0000","last_became_active":"2023-10-07T07:58:40.591769+0000","last_became_peered":"2023-10-07T07:58:40.591769+0000","last_unstale":"2023-10-07T07:58:40.595075+0000","last_undegraded":"2023-10-07T07:58:40.595075+0000","last_fullsized":"2023-10-07T07:58:40.595075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:02:25.944882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.576988+0000","last_change":"2023-10-07T07:58:39.584951+0000","last_active":"2023-10-07T07:58:40.576988+0000","last_peered":"2023-10-07T07:58:40.576988+0000","last_clean":"2023-10-07T07:58:40.576988+0000","last_became_active":"2023-10-07T07:58:39.584708+0000","last_became_peered":"2023-10-07T07:58:39.584708+0000","last_unstale":"2023-10-07T07:58:40.576988+0000","last_undegraded":"2023-10-07T07:58:40.576988+0000","last_fullsized":"2023-10-07T07:58:40.576988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:02:50.444828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.594422+0000","last_change":"2023-10-07T07:58:40.590753+0000","last_active":"2023-10-07T07:58:40.594422+0000","last_peered":"2023-10-07T07:58:40.594422+0000","last_clean":"2023-10-07T07:58:40.594422+0000","last_became_active":"2023-10-07T07:58:40.590475+0000","last_became_peered":"2023-10-07T07:58:40.590475+0000","last_unstale":"2023-10-07T07:58:40.594422+0000","last_undegraded":"2023-10-07T07:58:40.594422+0000","last_fullsized":"2023-10-07T07:58:40.594422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:14:18.503786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.583970+0000","last_change":"2023-10-07T07:58:39.593877+0000","last_active":"2023-10-07T07:58:40.583970+0000","last_peered":"2023-10-07T07:58:40.583970+0000","last_clean":"2023-10-07T07:58:40.583970+0000","last_became_active":"2023-10-07T07:58:39.593573+0000","last_became_peered":"2023-10-07T07:58:39.593573+0000","last_unstale":"2023-10-07T07:58:40.583970+0000","last_undegraded":"2023-10-07T07:58:40.583970+0000","last_fullsized":"2023-10-07T07:58:40.583970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:30.410237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.605823+0000","last_change":"2023-10-07T07:58:40.588788+0000","last_active":"2023-10-07T07:58:40.605823+0000","last_peered":"2023-10-07T07:58:40.605823+0000","last_clean":"2023-10-07T07:58:40.605823+0000","last_became_active":"2023-10-07T07:58:40.588280+0000","last_became_peered":"2023-10-07T07:58:40.588280+0000","last_unstale":"2023-10-07T07:58:40.605823+0000","last_undegraded":"2023-10-07T07:58:40.605823+0000","last_fullsized":"2023-10-07T07:58:40.605823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:50:18.856129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.579125+0000","last_change":"2023-10-07T07:58:39.583131+0000","last_active":"2023-10-07T07:58:40.579125+0000","last_peered":"2023-10-07T07:58:40.579125+0000","last_clean":"2023-10-07T07:58:40.579125+0000","last_became_active":"2023-10-07T07:58:39.582847+0000","last_became_peered":"2023-10-07T07:58:39.582847+0000","last_unstale":"2023-10-07T07:58:40.579125+0000","last_undegraded":"2023-10-07T07:58:40.579125+0000","last_fullsized":"2023-10-07T07:58:40.579125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:40:32.472195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.604017+0000","last_change":"2023-10-07T07:58:40.593610+0000","last_active":"2023-10-07T07:58:40.604017+0000","last_peered":"2023-10-07T07:58:40.604017+0000","last_clean":"2023-10-07T07:58:40.604017+0000","last_became_active":"2023-10-07T07:58:40.593257+0000","last_became_peered":"2023-10-07T07:58:40.593257+0000","last_unstale":"2023-10-07T07:58:40.604017+0000","last_undegraded":"2023-10-07T07:58:40.604017+0000","last_fullsized":"2023-10-07T07:58:40.604017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:39.562521+0000","last_clean_scrub_stamp":"2023-10-07T07:58:39.562521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:29:17.476884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.576673+0000","last_change":"2023-10-07T07:58:39.592025+0000","last_active":"2023-10-07T07:58:40.576673+0000","last_peered":"2023-10-07T07:58:40.576673+0000","last_clean":"2023-10-07T07:58:40.576673+0000","last_became_active":"2023-10-07T07:58:39.591387+0000","last_became_peered":"2023-10-07T07:58:39.591387+0000","last_unstale":"2023-10-07T07:58:40.576673+0000","last_undegraded":"2023-10-07T07:58:40.576673+0000","last_fullsized":"2023-10-07T07:58:40.576673+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:58:38.556623+0000","last_clean_scrub_stamp":"2023-10-07T07:58:38.556623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:09:00.660419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-07T07:58:40.588505+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T07:58:40.588505+0000","last_peered":"2023-10-07T07:58:40.588505+0000","last_clean":"2023-10-07T07:58:40.588505+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T07:58:40.588505+0000","last_undegraded":"2023-10-07T07:58:40.588505+0000","last_fullsized":"2023-10-07T07:58:40.588505+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":143706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":138932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":734122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":143774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":143718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.372}]},{"osd":1,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.40200000000000002}]},{"osd":4,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71099999999999997}]},{"osd":5,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72499999999999998}]},{"osd":7,"last update":"Sat Oct 7 07:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.79700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":144432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":138954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51500000000000001}]},{"osd":1,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.67100000000000004}]},{"osd":6,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.752}]},{"osd":7,"last update":"Sat Oct 7 07:58:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":734606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T07:58:45.185 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-07T07:58:45.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-07T07:58:45.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.360+0000 7f22889d7700 1 Processor -- start 2023-10-07T07:58:45.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.360+0000 7f22889d7700 1 -- start start 2023-10-07T07:58:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.361+0000 7f22889d7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2284156f30 0x7f22840b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.361+0000 7f22889d7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b0190 0x7f22840b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.361+0000 7f22889d7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22840b18d0 con 0x7f22840b22a0 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.361+0000 7f22889d7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22840b0fa0 con 0x7f2284156f30 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.361+0000 7f22889d7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22840b1280 con 0x7f22840b0190 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.361+0000 7f228259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2284156f30 0x7f22840b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.361+0000 7f228259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2284156f30 0x7f22840b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56386/0 (socket says 172.21.15.7:56386) 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.361+0000 7f228259c700 1 -- 172.21.15.7:0/2611149299 learned_addr learned my addr 172.21.15.7:0/2611149299 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.362+0000 7f2282d9d700 1 --2- 172.21.15.7:0/2611149299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b0190 0x7f22840b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.362+0000 7f228259c700 1 -- 172.21.15.7:0/2611149299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b0190 msgr2=0x7f22840b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.362+0000 7f228259c700 1 --2- 172.21.15.7:0/2611149299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b0190 0x7f22840b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.362+0000 7f228259c700 1 -- 172.21.15.7:0/2611149299 >> v1:172.21.15.172:6789/0 conn(0x7f22840b22a0 legacy=0x7f22840afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.362+0000 7f228259c700 1 -- 172.21.15.7:0/2611149299 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22840b1560 con 0x7f2284156f30 2023-10-07T07:58:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.362+0000 7f228259c700 1 --2- 172.21.15.7:0/2611149299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 0x7f22840b1d60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f226c0099b0 tx=0x7f226c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e176b67b6bb569f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.362+0000 7f228159a700 1 -- 172.21.15.7:0/2611149299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00b860 con 0x7f2284156f30 2023-10-07T07:58:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.362+0000 7f228159a700 1 -- 172.21.15.7:0/2611149299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c00b9c0 con 0x7f2284156f30 2023-10-07T07:58:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.362+0000 7f228159a700 1 -- 172.21.15.7:0/2611149299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00bd60 con 0x7f2284156f30 2023-10-07T07:58:45.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.363+0000 7f22889d7700 1 -- 172.21.15.7:0/2611149299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 msgr2=0x7f22840b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.363+0000 7f22889d7700 1 --2- 172.21.15.7:0/2611149299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 0x7f22840b1d60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f226c0099b0 tx=0x7f226c009250 comp rx=0 tx=0).stop 2023-10-07T07:58:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.363+0000 7f22889d7700 1 -- 172.21.15.7:0/2611149299 shutdown_connections 2023-10-07T07:58:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.363+0000 7f22889d7700 1 --2- 172.21.15.7:0/2611149299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b0190 0x7f22840b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.363+0000 7f22889d7700 1 --2- 172.21.15.7:0/2611149299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 0x7f22840b1d60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.363+0000 7f22889d7700 1 -- 172.21.15.7:0/2611149299 >> 172.21.15.7:0/2611149299 conn(0x7f228406d350 msgr2=0x7f228406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.363+0000 7f22889d7700 1 -- 172.21.15.7:0/2611149299 shutdown_connections 2023-10-07T07:58:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.363+0000 7f22889d7700 1 -- 172.21.15.7:0/2611149299 wait complete. 2023-10-07T07:58:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.364+0000 7f22889d7700 1 Processor -- start 2023-10-07T07:58:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.364+0000 7f22889d7700 1 -- start start 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f22889d7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22840b0190 0x7f228416c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f22889d7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b22a0 0x7f2284169690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f22889d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 0x7f2284169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f22889d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228416caa0 con 0x7f2284156f30 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f22889d7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2284068820 con 0x7f22840b0190 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f22889d7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2284068de0 con 0x7f22840b22a0 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f2282d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 0x7f2284169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f2282d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 0x7f2284169bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56394/0 (socket says 172.21.15.7:56394) 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f2282d9d700 1 -- 172.21.15.7:0/77632234 learned_addr learned my addr 172.21.15.7:0/77632234 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f228259c700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22840b0190 0x7f228416c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f2281d9b700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b22a0 0x7f2284169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f2282d9d700 1 -- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b22a0 msgr2=0x7f2284169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f2282d9d700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b22a0 0x7f2284169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f2282d9d700 1 -- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22840b0190 msgr2=0x7f228416c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f2282d9d700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22840b0190 0x7f228416c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.365+0000 7f2282d9d700 1 -- 172.21.15.7:0/77632234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0094f0 con 0x7f2284156f30 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.366+0000 7f2282d9d700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 0x7f2284169bd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fa0 tx=0x7f227c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.366+0000 7f227a7fc700 1 -- 172.21.15.7:0/77632234 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00ebc0 con 0x7f2284156f30 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.366+0000 7f227a7fc700 1 -- 172.21.15.7:0/77632234 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c00ed20 con 0x7f2284156f30 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.366+0000 7f227a7fc700 1 -- 172.21.15.7:0/77632234 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c014670 con 0x7f2284156f30 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.366+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228416a1c0 con 0x7f2284156f30 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.366+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2284068a70 con 0x7f2284156f30 2023-10-07T07:58:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.367+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228404c3c0 con 0x7f2284156f30 2023-10-07T07:58:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.367+0000 7f227a7fc700 1 -- 172.21.15.7:0/77632234 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f227c0149e0 con 0x7f2284156f30 2023-10-07T07:58:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.368+0000 7f227a7fc700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2264066f30 0x7f22640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.368+0000 7f227a7fc700 1 -- 172.21.15.7:0/77632234 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f227c084b00 con 0x7f2284156f30 2023-10-07T07:58:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.368+0000 7f228259c700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2264066f30 0x7f22640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.368+0000 7f228259c700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2264066f30 0x7f22640693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f226c0099b0 tx=0x7f226c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:45.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.372+0000 7f227a7fc700 1 -- 172.21.15.7:0/77632234 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f227c086e60 con 0x7f2284156f30 2023-10-07T07:58:45.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.655+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f22840b3d50 con 0x7f2284156f30 2023-10-07T07:58:45.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.659+0000 7f227a7fc700 1 -- 172.21.15.7:0/77632234 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f227c051cd0 con 0x7f2284156f30 2023-10-07T07:58:45.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2264066f30 msgr2=0x7f22640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2264066f30 0x7f22640693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f226c0099b0 tx=0x7f226c017000 comp rx=0 tx=0).stop 2023-10-07T07:58:45.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 msgr2=0x7f2284169bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 0x7f2284169bd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fa0 tx=0x7f227c009900 comp rx=0 tx=0).stop 2023-10-07T07:58:45.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 shutdown_connections 2023-10-07T07:58:45.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2264066f30 0x7f22640693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284156f30 0x7f2284169bd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22840b0190 0x7f228416c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 --2- 172.21.15.7:0/77632234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22840b22a0 0x7f2284169690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 >> 172.21.15.7:0/77632234 conn(0x7f228406d350 msgr2=0x7f22840cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:45.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 shutdown_connections 2023-10-07T07:58:45.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.661+0000 7f22889d7700 1 -- 172.21.15.7:0/77632234 wait complete. 2023-10-07T07:58:45.664 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2023-10-07T07:58:45.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:58:45.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.806+0000 7f53222a9700 1 Processor -- start 2023-10-07T07:58:45.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.806+0000 7f53222a9700 1 -- start start 2023-10-07T07:58:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.807+0000 7f53222a9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f531c159310 0x7f531c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.807+0000 7f53222a9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c068100 con 0x7f531c0ced30 2023-10-07T07:58:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.807+0000 7f53222a9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c068350 con 0x7f531c1596e0 2023-10-07T07:58:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.807+0000 7f53222a9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c068630 con 0x7f531c159310 2023-10-07T07:58:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.807+0000 7f531bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f531c159310 0x7f531c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.807+0000 7f531b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f531c0ced30 0x7f531c0c3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58142/0 (socket says 172.21.15.7:58142) 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.807+0000 7f531b7fe700 1 -- 172.21.15.7:0/1466248337 learned_addr learned my addr 172.21.15.7:0/1466248337 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.808+0000 7f531affd700 1 -- 172.21.15.7:0/1466248337 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560422889 0 0) 0x7f531c068100 con 0x7f531c0ced30 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.808+0000 7f531affd700 1 -- 172.21.15.7:0/1466248337 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5304003570 con 0x7f531c0ced30 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.808+0000 7f531bfff700 1 -- 172.21.15.7:0/1466248337 >> v1:172.21.15.172:6789/0 conn(0x7f531c1596e0 legacy=0x7f531c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.808+0000 7f531bfff700 1 -- 172.21.15.7:0/1466248337 >> v1:172.21.15.7:6789/0 conn(0x7f531c0ced30 legacy=0x7f531c0c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.808+0000 7f531bfff700 1 -- 172.21.15.7:0/1466248337 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c068910 con 0x7f531c159310 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.808+0000 7f531bfff700 1 --2- 172.21.15.7:0/1466248337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c159310 0x7f531c0c86e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f530c0098e0 tx=0x7f530c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1c228071cb1dac server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.808+0000 7f531affd700 1 -- 172.21.15.7:0/1466248337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00b640 con 0x7f531c159310 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.808+0000 7f531affd700 1 -- 172.21.15.7:0/1466248337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c00b7a0 con 0x7f531c159310 2023-10-07T07:58:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.808+0000 7f531affd700 1 -- 172.21.15.7:0/1466248337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00bb20 con 0x7f531c159310 2023-10-07T07:58:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.809+0000 7f53222a9700 1 -- 172.21.15.7:0/1466248337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c159310 msgr2=0x7f531c0c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.809+0000 7f53222a9700 1 --2- 172.21.15.7:0/1466248337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c159310 0x7f531c0c86e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f530c0098e0 tx=0x7f530c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.809+0000 7f53222a9700 1 -- 172.21.15.7:0/1466248337 shutdown_connections 2023-10-07T07:58:45.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.809+0000 7f53222a9700 1 --2- 172.21.15.7:0/1466248337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c159310 0x7f531c0c86e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.809+0000 7f53222a9700 1 -- 172.21.15.7:0/1466248337 >> 172.21.15.7:0/1466248337 conn(0x7f531c067ac0 msgr2=0x7f531c0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.809+0000 7f53222a9700 1 -- 172.21.15.7:0/1466248337 shutdown_connections 2023-10-07T07:58:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.809+0000 7f53222a9700 1 -- 172.21.15.7:0/1466248337 wait complete. 2023-10-07T07:58:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.810+0000 7f53222a9700 1 Processor -- start 2023-10-07T07:58:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.811+0000 7f53222a9700 1 -- start start 2023-10-07T07:58:45.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f53222a9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f531c159310 0x7f531c0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f53222a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c1596e0 0x7f531c169e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f53222a9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f531c0b1380 0x7f531c16a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f53222a9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c151660 con 0x7f531c1596e0 2023-10-07T07:58:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f53222a9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c0b3890 con 0x7f531c0b1380 2023-10-07T07:58:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f53222a9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c0b3b70 con 0x7f531c159310 2023-10-07T07:58:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c1596e0 0x7f531c169e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c1596e0 0x7f531c169e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56420/0 (socket says 172.21.15.7:56420) 2023-10-07T07:58:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531b7fe700 1 -- 172.21.15.7:0/2735781363 learned_addr learned my addr 172.21.15.7:0/2735781363 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:45.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531bfff700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f531c159310 0x7f531c0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f5320846700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f531c0b1380 0x7f531c16a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531b7fe700 1 -- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f531c159310 msgr2=0x7f531c0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531b7fe700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f531c159310 0x7f531c0b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531b7fe700 1 -- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f531c0b1380 msgr2=0x7f531c16a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531b7fe700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f531c0b1380 0x7f531c16a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531b7fe700 1 -- 172.21.15.7:0/2735781363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c009420 con 0x7f531c1596e0 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.812+0000 7f531b7fe700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c1596e0 0x7f531c169e90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f531000c590 tx=0x7f5310007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.813+0000 7f5303fff700 1 -- 172.21.15.7:0/2735781363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310005b30 con 0x7f531c1596e0 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.813+0000 7f5303fff700 1 -- 172.21.15.7:0/2735781363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5310005cb0 con 0x7f531c1596e0 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.813+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c1512f0 con 0x7f531c1596e0 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.813+0000 7f5303fff700 1 -- 172.21.15.7:0/2735781363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310007320 con 0x7f531c1596e0 2023-10-07T07:58:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.813+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c0b34c0 con 0x7f531c1596e0 2023-10-07T07:58:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.814+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c151810 con 0x7f531c1596e0 2023-10-07T07:58:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.814+0000 7f5303fff700 1 -- 172.21.15.7:0/2735781363 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f531001b3b0 con 0x7f531c1596e0 2023-10-07T07:58:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.815+0000 7f5303fff700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5308066f30 0x7f53080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.815+0000 7f5303fff700 1 -- 172.21.15.7:0/2735781363 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5310083960 con 0x7f531c1596e0 2023-10-07T07:58:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.815+0000 7f531bfff700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5308066f30 0x7f53080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.815+0000 7f531bfff700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5308066f30 0x7f53080693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f530c0093f0 tx=0x7f530c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:45.819+0000 7f5303fff700 1 -- 172.21.15.7:0/2735781363 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f531c151810 con 0x7f531c1596e0 2023-10-07T07:58:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.068+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f531c151140 con 0x7f531c1596e0 2023-10-07T07:58:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.068+0000 7f5303fff700 1 -- 172.21.15.7:0/2735781363 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f5310017020 con 0x7f531c1596e0 2023-10-07T07:58:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:58:46.070 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/580346028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":580346028},{"type":"v1","addr":"172.21.15.172:6837","nonce":580346028}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3012863722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3012863722},{"type":"v1","addr":"172.21.15.7:6835","nonce":3012863722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4947,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3733075096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3733075096},{"type":"v1","addr":"172.21.15.172:6836","nonce":3733075096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:58:40.003670+0000","modified":"2023-10-07T07:58:43.589885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4959":{"gid":4959,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/786560894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":786560894},{"type":"v1","addr":"172.21.15.7:6834","nonce":786560894}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T07:58:46.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5308066f30 msgr2=0x7f53080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5308066f30 0x7f53080693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f530c0093f0 tx=0x7f530c017000 comp rx=0 tx=0).stop 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c1596e0 msgr2=0x7f531c169e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c1596e0 0x7f531c169e90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f531000c590 tx=0x7f5310007e00 comp rx=0 tx=0).stop 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 shutdown_connections 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5308066f30 0x7f53080693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f531c0b1380 0x7f531c16a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f531c159310 0x7f531c0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 --2- 172.21.15.7:0/2735781363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f531c1596e0 0x7f531c169e90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 >> 172.21.15.7:0/2735781363 conn(0x7f531c067ac0 msgr2=0x7f531c153970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 shutdown_connections 2023-10-07T07:58:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.071+0000 7f53222a9700 1 -- 172.21.15.7:0/2735781363 wait complete. 2023-10-07T07:58:46.074 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-10-07T07:58:46.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-07T07:58:40.003670+0000', 'modified': '2023-10-07T07:58:43.589885+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4959}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4959': {'gid': 4959, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.7:6834/786560894', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6832', 'nonce': 786560894}, {'type': 'v1', 'addr': '172.21.15.7:6834', 'nonce': 786560894}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-07T07:58:46.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-07T07:58:46.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-07T07:58:46.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.632+0000 7fb223bdc700 1 Processor -- start 2023-10-07T07:58:46.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.632+0000 7fb223bdc700 1 -- start start 2023-10-07T07:58:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb223bdc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 0x7fb21c06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb223bdc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb21c159690 0x7fb21c06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb223bdc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb21c0cab80 0x7fb21c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb223bdc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21c06bb70 con 0x7fb21c159690 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb223bdc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21c0cda10 con 0x7fb21c0cab80 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb223bdc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c06af60 con 0x7fb21c1592c0 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221177700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb21c159690 0x7fb21c06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221177700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb21c159690 0x7fb21c06a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56434/0 (socket says 172.21.15.7:56434) 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221177700 1 -- 172.21.15.7:0/3245583562 learned_addr learned my addr 172.21.15.7:0/3245583562 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221978700 1 --2- 172.21.15.7:0/3245583562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 0x7fb21c06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb222179700 1 --2- 172.21.15.7:0/3245583562 >> v2:172.21.15.172:3300/0 conn(0x7fb21c0cab80 0x7fb21c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221177700 1 -- 172.21.15.7:0/3245583562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 msgr2=0x7fb21c06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221177700 1 --2- 172.21.15.7:0/3245583562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 0x7fb21c06a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221177700 1 -- 172.21.15.7:0/3245583562 >> v2:172.21.15.172:3300/0 conn(0x7fb21c0cab80 msgr2=0x7fb21c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221177700 1 --2- 172.21.15.7:0/3245583562 >> v2:172.21.15.172:3300/0 conn(0x7fb21c0cab80 0x7fb21c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221177700 1 -- 172.21.15.7:0/3245583562 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c06b240 con 0x7fb21c159690 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221978700 1 --2- 172.21.15.7:0/3245583562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 0x7fb21c06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb222179700 1 --2- 172.21.15.7:0/3245583562 >> v2:172.21.15.172:3300/0 conn(0x7fb21c0cab80 0x7fb21c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.633+0000 7fb221177700 1 --2- 172.21.15.7:0/3245583562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 0x7fb21c06a930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb21c06ae70 tx=0x7fb210009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322b295d16a542a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb220976700 1 -- 172.21.15.7:0/3245583562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000b860 con 0x7fb21c159690 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb220976700 1 -- 172.21.15.7:0/3245583562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21000b9c0 con 0x7fb21c159690 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb220976700 1 -- 172.21.15.7:0/3245583562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000bd60 con 0x7fb21c159690 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb223bdc700 1 -- 172.21.15.7:0/3245583562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 msgr2=0x7fb21c06a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb223bdc700 1 --2- 172.21.15.7:0/3245583562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 0x7fb21c06a930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb21c06ae70 tx=0x7fb210009250 comp rx=0 tx=0).stop 2023-10-07T07:58:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb223bdc700 1 -- 172.21.15.7:0/3245583562 shutdown_connections 2023-10-07T07:58:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb223bdc700 1 --2- 172.21.15.7:0/3245583562 >> v2:172.21.15.172:3300/0 conn(0x7fb21c0cab80 0x7fb21c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb223bdc700 1 --2- 172.21.15.7:0/3245583562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 0x7fb21c06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb223bdc700 1 --2- 172.21.15.7:0/3245583562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 0x7fb21c06a930 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb223bdc700 1 -- 172.21.15.7:0/3245583562 >> 172.21.15.7:0/3245583562 conn(0x7fb21c068bd0 msgr2=0x7fb21c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb223bdc700 1 -- 172.21.15.7:0/3245583562 shutdown_connections 2023-10-07T07:58:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.634+0000 7fb223bdc700 1 -- 172.21.15.7:0/3245583562 wait complete. 2023-10-07T07:58:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.636+0000 7fb223bdc700 1 Processor -- start 2023-10-07T07:58:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.636+0000 7fb223bdc700 1 -- start start 2023-10-07T07:58:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb223bdc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb21c0cab80 0x7fb21c16cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb223bdc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 0x7fb21c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb223bdc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 0x7fb21c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb223bdc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c0ac1a0 con 0x7fb21c159690 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb223bdc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c16d4f0 con 0x7fb21c0cab80 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb223bdc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c16d7d0 con 0x7fb21c1592c0 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb222179700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 0x7fb21c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb222179700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 0x7fb21c1478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56438/0 (socket says 172.21.15.7:56438) 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb222179700 1 -- 172.21.15.7:0/1737523940 learned_addr learned my addr 172.21.15.7:0/1737523940 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb221978700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb21c0cab80 0x7fb21c16cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb221177700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 0x7fb21c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb222179700 1 -- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 msgr2=0x7fb21c06ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb222179700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 0x7fb21c06ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb222179700 1 -- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb21c0cab80 msgr2=0x7fb21c16cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb222179700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb21c0cab80 0x7fb21c16cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.637+0000 7fb222179700 1 -- 172.21.15.7:0/1737523940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100094f0 con 0x7fb21c159690 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.638+0000 7fb222179700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 0x7fb21c1478c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009fc0 tx=0x7fb20c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.638+0000 7fb219ffb700 1 -- 172.21.15.7:0/1737523940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00dd40 con 0x7fb21c159690 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.638+0000 7fb219ffb700 1 -- 172.21.15.7:0/1737523940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c00f040 con 0x7fb21c159690 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.638+0000 7fb223bdc700 1 -- 172.21.15.7:0/1737523940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21c16dab0 con 0x7fb21c159690 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.638+0000 7fb219ffb700 1 -- 172.21.15.7:0/1737523940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c015690 con 0x7fb21c159690 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.638+0000 7fb223bdc700 1 -- 172.21.15.7:0/1737523940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21c16d170 con 0x7fb21c159690 2023-10-07T07:58:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.639+0000 7fb219ffb700 1 -- 172.21.15.7:0/1737523940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb20c01e3b0 con 0x7fb21c159690 2023-10-07T07:58:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.640+0000 7fb219ffb700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb208066f30 0x7fb2080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.640+0000 7fb219ffb700 1 -- 172.21.15.7:0/1737523940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb20c084a80 con 0x7fb21c159690 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.640+0000 7fb221978700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb208066f30 0x7fb2080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.641+0000 7fb221978700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb208066f30 0x7fb2080693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb214006fd0 tx=0x7fb214008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.642+0000 7fb22cef0b80 1 Processor -- start 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.642+0000 7fb22cef0b80 1 -- start start 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68d9a40 0x55f9d68d9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 0x55f9d68e2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68e3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d67ac310 con 0x55f9d68e2bc0 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d67e9130 con 0x55f9d68d9a40 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9d67e6af0 con 0x55f9d68bf530 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb222179700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68e3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb222179700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68e3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56444/0 (socket says 172.21.15.7:56444) 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb222179700 1 -- 172.21.15.7:0/2297091643 learned_addr learned my addr 172.21.15.7:0/2297091643 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb221978700 1 --2- 172.21.15.7:0/2297091643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68d9a40 0x55f9d68d9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb221177700 1 --2- 172.21.15.7:0/2297091643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 0x55f9d68e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb222179700 1 -- 172.21.15.7:0/2297091643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 msgr2=0x55f9d68e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb222179700 1 --2- 172.21.15.7:0/2297091643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 0x55f9d68e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb222179700 1 -- 172.21.15.7:0/2297091643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68d9a40 msgr2=0x55f9d68d9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb222179700 1 --2- 172.21.15.7:0/2297091643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68d9a40 0x55f9d68d9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.643+0000 7fb222179700 1 -- 172.21.15.7:0/2297091643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d66d50a0 con 0x55f9d68e2bc0 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb222179700 1 --2- 172.21.15.7:0/2297091643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68e3010 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb20c083050 tx=0x7fb20c082f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b03106e6083903 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb202ffd700 1 -- 172.21.15.7:0/2297091643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c055780 con 0x55f9d68e2bc0 2023-10-07T07:58:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb202ffd700 1 -- 172.21.15.7:0/2297091643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c058c00 con 0x55f9d68e2bc0 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb202ffd700 1 -- 172.21.15.7:0/2297091643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c05c950 con 0x55f9d68e2bc0 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb22cef0b80 1 -- 172.21.15.7:0/2297091643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 msgr2=0x55f9d68e3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/2297091643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68e3010 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb20c083050 tx=0x7fb20c082f00 comp rx=0 tx=0).stop 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb22cef0b80 1 -- 172.21.15.7:0/2297091643 shutdown_connections 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/2297091643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68e3010 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/2297091643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68d9a40 0x55f9d68d9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/2297091643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 0x55f9d68e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb22cef0b80 1 -- 172.21.15.7:0/2297091643 >> 172.21.15.7:0/2297091643 conn(0x55f9d68dce70 msgr2=0x55f9d68dd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb22cef0b80 1 -- 172.21.15.7:0/2297091643 shutdown_connections 2023-10-07T07:58:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.644+0000 7fb22cef0b80 1 -- 172.21.15.7:0/2297091643 wait complete. 2023-10-07T07:58:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.645+0000 7fb22cef0b80 1 -- start start 2023-10-07T07:58:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.645+0000 7fb22cef0b80 1 Processor -- start 2023-10-07T07:58:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 0x55f9d68ba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68d9a40 0x55f9d68b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68e2bc0 0x55f9d6893dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d67e28e0 con 0x55f9d68d9a40 2023-10-07T07:58:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d678e980 con 0x55f9d68e2bc0 2023-10-07T07:58:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9d678e620 con 0x55f9d68bf530 2023-10-07T07:58:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb22cef0b80 10 client.0 _ll_register_callbacks cb 0x55f9d65f1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb221177700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68d9a40 0x55f9d68b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb221177700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68d9a40 0x55f9d68b7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56458/0 (socket says 172.21.15.7:56458) 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb221177700 1 -- 172.21.15.7:0/1570446174 learned_addr learned my addr 172.21.15.7:0/1570446174 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb221978700 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 0x55f9d68ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb222179700 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68e2bc0 0x55f9d6893dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb221177700 1 -- 172.21.15.7:0/1570446174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 msgr2=0x55f9d68ba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb221177700 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 0x55f9d68ba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb221177700 1 -- 172.21.15.7:0/1570446174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68e2bc0 msgr2=0x55f9d6893dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb221177700 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68e2bc0 0x55f9d6893dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.647+0000 7fb221177700 1 -- 172.21.15.7:0/1570446174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21c16d170 con 0x55f9d68d9a40 2023-10-07T07:58:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb221177700 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68d9a40 0x55f9d68b7050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb210006fd0 tx=0x7fb210017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8981c0dba5346a8a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb1fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb1fbfff700 1 -- 172.21.15.7:0/1570446174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210020070 con 0x55f9d68d9a40 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb1fbfff700 1 -- 172.21.15.7:0/1570446174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb21000be10 con 0x55f9d68d9a40 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1570446174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9d6887560 con 0x55f9d68d9a40 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb1fbfff700 1 -- 172.21.15.7:0/1570446174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb210011840 con 0x55f9d68d9a40 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb22cef0b80 10 client.5124 fetch_fsmap learned FSMap version 16 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1570446174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f9d67b13c0 con 0x55f9d68d9a40 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb1fbfff700 1 -- 172.21.15.7:0/1570446174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9d6887560 con 0x55f9d68d9a40 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb1fbfff700 1 -- 172.21.15.7:0/1570446174 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb2100119a0 con 0x55f9d68d9a40 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb22cef0b80 10 client.5124 fetch_fsmap finished waiting for FSMap version 16 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb22cef0b80 10 client.5124 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-07T07:58:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb22cef0b80 20 client.5124 populate_metadata read hostname 'smithi007' 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55f9d68b8e80 0x55f9d68b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb22cef0b80 4 client.5124 mds_command: new command op to 4959 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1570446174 --> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9d68640e0 con 0x55f9d68b8e80 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.648+0000 7fb221978700 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55f9d68b8e80 0x55f9d68b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.649+0000 7fb1fbfff700 10 client.5124 ms_handle_connect on v2:172.21.15.7:6832/786560894 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.649+0000 7fb221978700 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55f9d68b8e80 0x55f9d68b9250 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea795337dcacc12 server_cookie=59b41196130e39f8 in_seq=0 out_seq=0 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.649+0000 7fb1fbfff700 1 -- 172.21.15.7:0/1570446174 <== mds.0 v2:172.21.15.7:6832/786560894 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f9d68640e0 con 0x55f9d68b8e80 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.649+0000 7fb1fbfff700 10 client.5124 handle_command_reply: tid=0 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.649+0000 7fb22cef0b80 1 client.5124 shutdown 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1570446174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68d9a40 msgr2=0x55f9d68b7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68d9a40 0x55f9d68b7050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb210006fd0 tx=0x7fb210017000 comp rx=0 tx=0).stop 2023-10-07T07:58:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1570446174 shutdown_connections 2023-10-07T07:58:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68e2bc0 0x55f9d6893dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68bf530 0x55f9d68ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55f9d68b8e80 0x55f9d68b9250 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1570446174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68d9a40 0x55f9d68b7050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb1fbfff700 0 client.5124 ms_handle_reset on v2:172.21.15.7:6832/786560894 2023-10-07T07:58:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1570446174 >> 172.21.15.7:0/1570446174 conn(0x55f9d68dce70 msgr2=0x55f9d688f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1570446174 shutdown_connections 2023-10-07T07:58:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1570446174 wait complete. 2023-10-07T07:58:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.650+0000 7fb22cef0b80 20 client.5124 trim_cache size 0 max 16384 2023-10-07T07:58:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.668+0000 7fb22cef0b80 1 Processor -- start 2023-10-07T07:58:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.668+0000 7fb22cef0b80 1 -- start start 2023-10-07T07:58:46.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 0x55f9d688fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68d9a40 0x55f9d68b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d667a1e0 con 0x55f9d68e2bc0 2023-10-07T07:58:46.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d68bf2f0 con 0x55f9d68bf530 2023-10-07T07:58:46.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9d68c1b20 con 0x55f9d68d9a40 2023-10-07T07:58:46.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb222179700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb222179700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56462/0 (socket says 172.21.15.7:56462) 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb222179700 1 -- 172.21.15.7:0/1061487182 learned_addr learned my addr 172.21.15.7:0/1061487182 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb222179700 1 -- 172.21.15.7:0/1061487182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68d9a40 msgr2=0x55f9d68b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb222179700 1 --2- 172.21.15.7:0/1061487182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68d9a40 0x55f9d68b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb222179700 1 -- 172.21.15.7:0/1061487182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 msgr2=0x55f9d688fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb221978700 1 --2- 172.21.15.7:0/1061487182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 0x55f9d688fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb222179700 1 --2- 172.21.15.7:0/1061487182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 0x55f9d688fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb222179700 1 -- 172.21.15.7:0/1061487182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d66491d0 con 0x55f9d68e2bc0 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.669+0000 7fb221978700 1 --2- 172.21.15.7:0/1061487182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 0x55f9d688fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb222179700 1 --2- 172.21.15.7:0/1061487182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bfaf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb20c053fa0 tx=0x7fb20c06d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b3ade5eded4d5f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb2037fe700 1 -- 172.21.15.7:0/1061487182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c082070 con 0x55f9d68e2bc0 2023-10-07T07:58:46.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb2037fe700 1 -- 172.21.15.7:0/1061487182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c05c950 con 0x55f9d68e2bc0 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb2037fe700 1 -- 172.21.15.7:0/1061487182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c0559b0 con 0x55f9d68e2bc0 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1061487182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 msgr2=0x55f9d68bfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1061487182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bfaf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb20c053fa0 tx=0x7fb20c06d000 comp rx=0 tx=0).stop 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1061487182 shutdown_connections 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1061487182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bfaf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1061487182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 0x55f9d688fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/1061487182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68d9a40 0x55f9d68b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1061487182 >> 172.21.15.7:0/1061487182 conn(0x55f9d68dce70 msgr2=0x55f9d6893b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1061487182 shutdown_connections 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.670+0000 7fb22cef0b80 1 -- 172.21.15.7:0/1061487182 wait complete. 2023-10-07T07:58:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.671+0000 7fb22cef0b80 1 -- start start 2023-10-07T07:58:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.671+0000 7fb22cef0b80 1 Processor -- start 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 0x55f9d6930890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68d9a40 0x55f9d68dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb22cef0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d6930dd0 con 0x55f9d68e2bc0 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d693f7c0 con 0x55f9d68bf530 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb22cef0b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9d68dda70 con 0x55f9d68d9a40 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb22cef0b80 10 client.0 _ll_register_callbacks cb 0x55f9d65f1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb222179700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb222179700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56464/0 (socket says 172.21.15.7:56464) 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.672+0000 7fb222179700 1 -- 172.21.15.7:0/774296659 learned_addr learned my addr 172.21.15.7:0/774296659 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:46.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb221978700 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 0x55f9d6930890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb222179700 1 -- 172.21.15.7:0/774296659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68d9a40 msgr2=0x55f9d68dd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb222179700 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68d9a40 0x55f9d68dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb222179700 1 -- 172.21.15.7:0/774296659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 msgr2=0x55f9d6930890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb222179700 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 0x55f9d6930890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb222179700 1 -- 172.21.15.7:0/774296659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9d66d50a0 con 0x55f9d68e2bc0 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb222179700 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bf900 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb20c055940 tx=0x7fb20c082000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a1e915cf172072 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb1fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb1fbfff700 1 -- 172.21.15.7:0/774296659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c054ea0 con 0x55f9d68e2bc0 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb1fbfff700 1 -- 172.21.15.7:0/774296659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb20c055000 con 0x55f9d68e2bc0 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb22cef0b80 1 -- 172.21.15.7:0/774296659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb20c058c00 con 0x55f9d68e2bc0 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb1fbfff700 1 -- 172.21.15.7:0/774296659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb20c05c6d0 con 0x55f9d68e2bc0 2023-10-07T07:58:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb22cef0b80 10 client.5130 fetch_fsmap learned FSMap version 16 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb22cef0b80 1 -- 172.21.15.7:0/774296659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f9d6622d60 con 0x55f9d68e2bc0 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb1fbfff700 1 -- 172.21.15.7:0/774296659 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c058c00 con 0x55f9d68e2bc0 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb1fbfff700 1 -- 172.21.15.7:0/774296659 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb20c063070 con 0x55f9d68e2bc0 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb22cef0b80 10 client.5130 fetch_fsmap finished waiting for FSMap version 16 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb22cef0b80 10 client.5130 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb22cef0b80 20 client.5130 populate_metadata read hostname 'smithi007' 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55f9d69337e0 0x55f9d6933bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb22cef0b80 4 client.5130 mds_command: new command op to 4959 tid=0[{"prefix": "status"}] 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.673+0000 7fb22cef0b80 1 -- 172.21.15.7:0/774296659 --> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f9d67858d0 con 0x55f9d69337e0 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb221978700 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55f9d69337e0 0x55f9d6933bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb1fbfff700 10 client.5130 ms_handle_connect on v2:172.21.15.7:6832/786560894 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb221978700 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55f9d69337e0 0x55f9d6933bb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6384a8d226bdf107 server_cookie=f3e7180ac21e6080 in_seq=0 out_seq=0 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb1fbfff700 1 -- 172.21.15.7:0/774296659 <== mds.0 v2:172.21.15.7:6832/786560894 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f9d67858d0 con 0x55f9d69337e0 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb1fbfff700 10 client.5130 handle_command_reply: tid=0 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 client.5130 shutdown 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 -- 172.21.15.7:0/774296659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 msgr2=0x55f9d68bf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bf900 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb20c055940 tx=0x7fb20c082000 comp rx=0 tx=0).stop 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 -- 172.21.15.7:0/774296659 shutdown_connections 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55f9d69337e0 0x55f9d6933bb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d68e2bc0 0x55f9d68bf900 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9d68bf530 0x55f9d6930890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 --2- 172.21.15.7:0/774296659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f9d68d9a40 0x55f9d68dd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb1fbfff700 0 client.5130 ms_handle_reset on v2:172.21.15.7:6832/786560894 2023-10-07T07:58:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 -- 172.21.15.7:0/774296659 >> 172.21.15.7:0/774296659 conn(0x55f9d68dce70 msgr2=0x55f9d68b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 -- 172.21.15.7:0/774296659 shutdown_connections 2023-10-07T07:58:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.674+0000 7fb22cef0b80 1 -- 172.21.15.7:0/774296659 wait complete. 2023-10-07T07:58:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb22cef0b80 20 client.5130 trim_cache size 0 max 16384 2023-10-07T07:58:46.682 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:58:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "c71d63e6-6de9-4139-83e7-b0bb7f6375d7", 2023-10-07T07:58:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2023-10-07T07:58:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4959, 2023-10-07T07:58:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2023-10-07T07:58:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2023-10-07T07:58:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2023-10-07T07:58:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.0969894330000001, 2023-10-07T07:58:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 16, 2023-10-07T07:58:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 33, 2023-10-07T07:58:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 33, 2023-10-07T07:58:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 14.961269890000001 2023-10-07T07:58:46.683 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:58:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 -- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb208066f30 msgr2=0x7fb2080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb208066f30 0x7fb2080693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb214006fd0 tx=0x7fb214008040 comp rx=0 tx=0).stop 2023-10-07T07:58:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 -- 172.21.15.7:0/1737523940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 msgr2=0x7fb21c1478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 0x7fb21c1478c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009fc0 tx=0x7fb20c0099b0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 -- 172.21.15.7:0/1737523940 shutdown_connections 2023-10-07T07:58:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb208066f30 0x7fb2080693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb21c159690 0x7fb21c1478c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb21c0cab80 0x7fb21c16cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 --2- 172.21.15.7:0/1737523940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb21c1592c0 0x7fb21c06ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 -- 172.21.15.7:0/1737523940 >> 172.21.15.7:0/1737523940 conn(0x7fb21c068bd0 msgr2=0x7fb21c0c2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 -- 172.21.15.7:0/1737523940 shutdown_connections 2023-10-07T07:58:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.675+0000 7fb223bdc700 1 -- 172.21.15.7:0/1737523940 wait complete. 2023-10-07T07:58:46.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T07:58:46.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.820+0000 7f18fa6c7700 1 Processor -- start 2023-10-07T07:58:46.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.820+0000 7f18fa6c7700 1 -- start start 2023-10-07T07:58:46.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.821+0000 7f18fa6c7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18f4140a80 0x7f18f406ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.821+0000 7f18fa6c7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 0x7f18f4066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.821+0000 7f18fa6c7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f18f4067530 0x7f18f406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.821+0000 7f18fa6c7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f406f250 con 0x7f18f4140a80 2023-10-07T07:58:46.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.821+0000 7f18fa6c7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f406e2c0 con 0x7f18f4067530 2023-10-07T07:58:46.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.821+0000 7f18fa6c7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f4067950 con 0x7f18f41582c0 2023-10-07T07:58:46.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.821+0000 7f18f3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18f4140a80 0x7f18f406ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.821+0000 7f18f3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18f4140a80 0x7f18f406ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56478/0 (socket says 172.21.15.7:56478) 2023-10-07T07:58:46.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.821+0000 7f18f3fff700 1 -- 172.21.15.7:0/1497144579 learned_addr learned my addr 172.21.15.7:0/1497144579 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:46.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f37fe700 1 --2- 172.21.15.7:0/1497144579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 0x7f18f4066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f8c64700 1 --2- 172.21.15.7:0/1497144579 >> v2:172.21.15.172:3300/0 conn(0x7f18f4067530 0x7f18f406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f3fff700 1 -- 172.21.15.7:0/1497144579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 msgr2=0x7f18f4066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f3fff700 1 --2- 172.21.15.7:0/1497144579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 0x7f18f4066ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f3fff700 1 -- 172.21.15.7:0/1497144579 >> v2:172.21.15.172:3300/0 conn(0x7f18f4067530 msgr2=0x7f18f406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:46.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f3fff700 1 --2- 172.21.15.7:0/1497144579 >> v2:172.21.15.172:3300/0 conn(0x7f18f4067530 0x7f18f406a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f3fff700 1 -- 172.21.15.7:0/1497144579 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40c7940 con 0x7f18f4140a80 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f3fff700 1 --2- 172.21.15.7:0/1497144579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 0x7f18f406ed10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f18f40c7580 tx=0x7f18e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89169fee34635cf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f2ffd700 1 -- 172.21.15.7:0/1497144579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e400b640 con 0x7f18f4140a80 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f2ffd700 1 -- 172.21.15.7:0/1497144579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e400b7a0 con 0x7f18f4140a80 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.822+0000 7f18f2ffd700 1 -- 172.21.15.7:0/1497144579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e400bb60 con 0x7f18f4140a80 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.823+0000 7f18fa6c7700 1 -- 172.21.15.7:0/1497144579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 msgr2=0x7f18f406ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.823+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/1497144579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 0x7f18f406ed10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f18f40c7580 tx=0x7f18e4009250 comp rx=0 tx=0).stop 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.823+0000 7f18fa6c7700 1 -- 172.21.15.7:0/1497144579 shutdown_connections 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.823+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/1497144579 >> v2:172.21.15.172:3300/0 conn(0x7f18f4067530 0x7f18f406a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.823+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/1497144579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 0x7f18f406ed10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.823+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/1497144579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 0x7f18f4066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.823+0000 7f18fa6c7700 1 -- 172.21.15.7:0/1497144579 >> 172.21.15.7:0/1497144579 conn(0x7f18f4068bd0 msgr2=0x7f18f40ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:46.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.823+0000 7f18fa6c7700 1 -- 172.21.15.7:0/1497144579 shutdown_connections 2023-10-07T07:58:46.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.823+0000 7f18fa6c7700 1 -- 172.21.15.7:0/1497144579 wait complete. 2023-10-07T07:58:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.824+0000 7f18fa6c7700 1 Processor -- start 2023-10-07T07:58:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.825+0000 7f18fa6c7700 1 -- start start 2023-10-07T07:58:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.825+0000 7f18fa6c7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18f4067530 0x7f18f40c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.825+0000 7f18fa6c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 0x7f18f4134cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.825+0000 7f18fa6c7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 0x7f18f40c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.825+0000 7f18fa6c7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40c6b50 con 0x7f18f4140a80 2023-10-07T07:58:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.825+0000 7f18fa6c7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40c6220 con 0x7f18f4067530 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.825+0000 7f18fa6c7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40c6500 con 0x7f18f41582c0 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.825+0000 7f18f37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 0x7f18f4134cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 0x7f18f4134cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56484/0 (socket says 172.21.15.7:56484) 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f37fe700 1 -- 172.21.15.7:0/3122893158 learned_addr learned my addr 172.21.15.7:0/3122893158 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f3fff700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18f4067530 0x7f18f40c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f8c64700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 0x7f18f40c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f37fe700 1 -- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 msgr2=0x7f18f40c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f37fe700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 0x7f18f40c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f37fe700 1 -- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18f4067530 msgr2=0x7f18f40c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f37fe700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18f4067530 0x7f18f40c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f37fe700 1 -- 172.21.15.7:0/3122893158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e40094f0 con 0x7f18f4140a80 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18f37fe700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 0x7f18f4134cf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e930 tx=0x7f18dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18ebfff700 1 -- 172.21.15.7:0/3122893158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00eed0 con 0x7f18f4140a80 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18ebfff700 1 -- 172.21.15.7:0/3122893158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc00c850 con 0x7f18f4140a80 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f40c67e0 con 0x7f18f4140a80 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18ebfff700 1 -- 172.21.15.7:0/3122893158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00cbf0 con 0x7f18f4140a80 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.826+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f40c5ed0 con 0x7f18f4140a80 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.827+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f4140310 con 0x7f18f4140a80 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.828+0000 7f18ebfff700 1 -- 172.21.15.7:0/3122893158 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18dc013470 con 0x7f18f4140a80 2023-10-07T07:58:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.828+0000 7f18ebfff700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f18d4066f30 0x7f18d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.828+0000 7f18ebfff700 1 -- 172.21.15.7:0/3122893158 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f18dc084b10 con 0x7f18f4140a80 2023-10-07T07:58:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.828+0000 7f18f3fff700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f18d4066f30 0x7f18d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.829+0000 7f18f3fff700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f18d4066f30 0x7f18d40693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18f40c7580 tx=0x7f18e4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:46.833+0000 7f18ebfff700 1 -- 172.21.15.7:0/3122893158 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18dc051d80 con 0x7f18f4140a80 2023-10-07T07:58:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.081+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f18f40c6970 con 0x7f18f4140a80 2023-10-07T07:58:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.081+0000 7f18ebfff700 1 -- 172.21.15.7:0/3122893158 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f18dc017070 con 0x7f18f4140a80 2023-10-07T07:58:47.083 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T07:58:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.083+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f18d4066f30 msgr2=0x7f18d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.083+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f18d4066f30 0x7f18d40693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18f40c7580 tx=0x7f18e4017000 comp rx=0 tx=0).stop 2023-10-07T07:58:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.083+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 msgr2=0x7f18f4134cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.083+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 0x7f18f4134cf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e930 tx=0x7f18dc0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.083+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 shutdown_connections 2023-10-07T07:58:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.083+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f18d4066f30 0x7f18d40693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.083+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18f41582c0 0x7f18f40c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.083+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18f4067530 0x7f18f40c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.083+0000 7f18fa6c7700 1 --2- 172.21.15.7:0/3122893158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f4140a80 0x7f18f4134cf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:47.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.084+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 >> 172.21.15.7:0/3122893158 conn(0x7f18f4068bd0 msgr2=0x7f18f4135550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:47.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.084+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 shutdown_connections 2023-10-07T07:58:47.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:47.084+0000 7f18fa6c7700 1 -- 172.21.15.7:0/3122893158 wait complete. 2023-10-07T07:58:47.098 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T07:58:47.098 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T07:58:47.098 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-07T07:58:47.098 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-07T07:58:47.098 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:47.098 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T07:58:47.098 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T07:58:47.099 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T07:58:47.099 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T07:58:47.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:47.099 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-07T07:58:47.120 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2023-10-07T07:58:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42408sec preferred_lft 42408sec 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:3a:d8:7f:89:ca brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f00c:5eff:fefc:98cd/64 scope link 2023-10-07T07:58:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:47.123 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:58:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 62:07:64:37:78:fc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T07:58:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6007:64ff:fe37:78fc/64 scope link 2023-10-07T07:58:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:47.123 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:58:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:3a:d8:7f:89:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T07:58:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::43a:d8ff:fe7f:89ca/64 scope link 2023-10-07T07:58:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:47.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:47.124 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T07:58:47.192 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-07T07:58:47.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:47.194 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-07T07:58:47.259 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-07T07:58:47.260 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T07:58:47.260 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-07T07:58:47.260 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-07T07:58:47.260 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T07:58:47.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:47.260 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:58:47.261 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:58:47.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T07:58:47.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T07:58:47.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T07:58:47.261 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:58:47.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:47.383 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-07T07:58:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-07T07:58:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T07:58:47.405 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T07:58:47.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:47.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2023-10-07T07:58:47.481 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-07T07:58:47.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:47.481 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:47.545 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-07T07:58:47.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:47.545 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:47.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T07:58:47.684 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:47.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:47.760 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:47.760 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:47.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:47.761 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T07:58:47.824 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T07:58:47.825 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-07T07:58:47.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T07:58:47.903 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:47.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:47.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-10-07T07:58:47.968+0000 7f39a14e5540 -1 init, newargv = 0x55e9f95d6dd0 newargc=15 2023-10-07T07:58:47.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[133382]: starting ceph client 2023-10-07T07:58:47.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[133382]: starting fuse 2023-10-07T07:58:47.975 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:47.976 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:47.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:47.977 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T07:58:48.038 INFO:teuthology.orchestra.run.smithi007.stdout:47 2023-10-07T07:58:48.039 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-07T07:58:48.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:48.039 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-07T07:58:48.039 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-07T07:58:48.039 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T07:58:48.039 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-07T07:58:48.039 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:48.040 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-07T07:58:48.040 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T07:58:48.040 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-07T07:58:48.040 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T07:58:48.040 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:48.040 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-07T07:58:48.040 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-07T07:58:48.040 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-07T07:58:48.040 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T07:58:48.041 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:48.042 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2023-10-07T07:58:48.042 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T07:58:48.178 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.133382.asok 2023-10-07T07:58:48.359 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.133382.asok 2023-10-07T07:58:48.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:48.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.133382.asok status 2023-10-07T07:58:48.534 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "133382", 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-07T07:58:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5142, 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 5142 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 377970135 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 377970135 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.5142 192.168.0.1:0/377970135", 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/377970135", 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 16, 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 33, 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-07T07:58:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-07T07:58:48.538 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:58:48.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:48.546 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-07T07:58:48.546 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:48.566 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-07T07:58:48.567 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:48.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:48.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-07T07:58:48.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T07:58:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.820+0000 7f0470eac700 1 Processor -- start 2023-10-07T07:58:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.820+0000 7f0470eac700 1 -- start start 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.821+0000 7f0470eac700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f046c0c9b50 0x7f046c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.821+0000 7f0470eac700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f046c0aed20 0x7f046c157380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.821+0000 7f0470eac700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046c0b1ed0 con 0x7f046c0c9f20 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.821+0000 7f0470eac700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046c0b18a0 con 0x7f046c0c9b50 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.821+0000 7f0470eac700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046c068390 con 0x7f046c0aed20 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.821+0000 7f046a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f046c0c9b50 0x7f046c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.821+0000 7f0469d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f046c0c9f20 0x7f046c156c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58148/0 (socket says 172.21.15.7:58148) 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.821+0000 7f0469d9b700 1 -- 172.21.15.7:0/1253039976 learned_addr learned my addr 172.21.15.7:0/1253039976 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046ad9d700 1 --2- 172.21.15.7:0/1253039976 >> v2:172.21.15.172:3300/0 conn(0x7f046c0aed20 0x7f046c157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046a59c700 1 -- 172.21.15.7:0/1253039976 >> v2:172.21.15.172:3300/0 conn(0x7f046c0aed20 msgr2=0x7f046c157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046a59c700 1 --2- 172.21.15.7:0/1253039976 >> v2:172.21.15.172:3300/0 conn(0x7f046c0aed20 0x7f046c157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046a59c700 1 -- 172.21.15.7:0/1253039976 >> v1:172.21.15.7:6789/0 conn(0x7f046c0c9f20 legacy=0x7f046c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046a59c700 1 -- 172.21.15.7:0/1253039976 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c068670 con 0x7f046c0c9b50 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046959a700 1 -- 172.21.15.7:0/1253039976 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100369745 0 0) 0x7f046c0b1ed0 con 0x7f046c0c9f20 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046a59c700 1 --2- 172.21.15.7:0/1253039976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 0x7f046c067b40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f046c0ca390 tx=0x7f04540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23f2ef05b4b670c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046959a700 1 -- 172.21.15.7:0/1253039976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045400b6b0 con 0x7f046c0c9b50 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046959a700 1 -- 172.21.15.7:0/1253039976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045400b810 con 0x7f046c0c9b50 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f046959a700 1 -- 172.21.15.7:0/1253039976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045400bbb0 con 0x7f046c0c9b50 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.822+0000 7f0470eac700 1 -- 172.21.15.7:0/1253039976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 msgr2=0x7f046c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.823+0000 7f0470eac700 1 --2- 172.21.15.7:0/1253039976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 0x7f046c067b40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f046c0ca390 tx=0x7f04540091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.823+0000 7f0470eac700 1 -- 172.21.15.7:0/1253039976 shutdown_connections 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.823+0000 7f0470eac700 1 --2- 172.21.15.7:0/1253039976 >> v2:172.21.15.172:3300/0 conn(0x7f046c0aed20 0x7f046c157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.823+0000 7f0470eac700 1 --2- 172.21.15.7:0/1253039976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 0x7f046c067b40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:48.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.823+0000 7f0470eac700 1 -- 172.21.15.7:0/1253039976 >> 172.21.15.7:0/1253039976 conn(0x7f046c0cd1a0 msgr2=0x7f046c0c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:48.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.823+0000 7f0470eac700 1 -- 172.21.15.7:0/1253039976 shutdown_connections 2023-10-07T07:58:48.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.823+0000 7f0470eac700 1 -- 172.21.15.7:0/1253039976 wait complete. 2023-10-07T07:58:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.824+0000 7f0470eac700 1 Processor -- start 2023-10-07T07:58:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.824+0000 7f0470eac700 1 -- start start 2023-10-07T07:58:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0470eac700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f046c0aed20 0x7f046c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0470eac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 0x7f046c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0470eac700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f046c0c9f20 0x7f046c15d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0470eac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c167d30 con 0x7f046c0c9b50 2023-10-07T07:58:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0470eac700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c15acf0 con 0x7f046c0aed20 2023-10-07T07:58:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0470eac700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046c15a450 con 0x7f046c0c9f20 2023-10-07T07:58:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0469d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 0x7f046c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0469d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 0x7f046c15d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56512/0 (socket says 172.21.15.7:56512) 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0469d9b700 1 -- 172.21.15.7:0/3667476815 learned_addr learned my addr 172.21.15.7:0/3667476815 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f046ad9d700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f046c0c9f20 0x7f046c15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f046a59c700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f046c0aed20 0x7f046c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0469d9b700 1 -- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f046c0c9f20 msgr2=0x7f046c15d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0469d9b700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f046c0c9f20 0x7f046c15d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0469d9b700 1 -- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f046c0aed20 msgr2=0x7f046c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0469d9b700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f046c0aed20 0x7f046c159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0469d9b700 1 -- 172.21.15.7:0/3667476815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0454009420 con 0x7f046c0c9b50 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.825+0000 7f0469d9b700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 0x7f046c15d3f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c5b0 tx=0x7f045c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.826+0000 7f04627fc700 1 -- 172.21.15.7:0/3667476815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c005b30 con 0x7f046c0c9b50 2023-10-07T07:58:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.826+0000 7f04627fc700 1 -- 172.21.15.7:0/3667476815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045c005c90 con 0x7f046c0c9b50 2023-10-07T07:58:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.826+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046c15a730 con 0x7f046c0c9b50 2023-10-07T07:58:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.826+0000 7f04627fc700 1 -- 172.21.15.7:0/3667476815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c007300 con 0x7f046c0c9b50 2023-10-07T07:58:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.826+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046c15aac0 con 0x7f046c0c9b50 2023-10-07T07:58:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.827+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046c168170 con 0x7f046c0c9b50 2023-10-07T07:58:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.827+0000 7f04627fc700 1 -- 172.21.15.7:0/3667476815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f045c01b3b0 con 0x7f046c0c9b50 2023-10-07T07:58:48.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.827+0000 7f04627fc700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f044c066f30 0x7f044c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:48.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.828+0000 7f04627fc700 1 -- 172.21.15.7:0/3667476815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f045c0839c0 con 0x7f046c0c9b50 2023-10-07T07:58:48.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.828+0000 7f046a59c700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f044c066f30 0x7f044c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:48.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.828+0000 7f046a59c700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f044c066f30 0x7f044c0693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f04540093f0 tx=0x7f0454017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:48.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:48.832+0000 7f04627fc700 1 -- 172.21.15.7:0/3667476815 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f046c168170 con 0x7f046c0c9b50 2023-10-07T07:58:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.083+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f046c168170 con 0x7f046c0c9b50 2023-10-07T07:58:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.083+0000 7f04627fc700 1 -- 172.21.15.7:0/3667476815 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f046c168170 con 0x7f046c0c9b50 2023-10-07T07:58:49.084 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T07:58:49.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.085+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f044c066f30 msgr2=0x7f044c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:49.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.085+0000 7f0470eac700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f044c066f30 0x7f044c0693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f04540093f0 tx=0x7f0454017000 comp rx=0 tx=0).stop 2023-10-07T07:58:49.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.085+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 msgr2=0x7f046c15d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:49.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.085+0000 7f0470eac700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 0x7f046c15d3f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c5b0 tx=0x7f045c001070 comp rx=0 tx=0).stop 2023-10-07T07:58:49.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.086+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 shutdown_connections 2023-10-07T07:58:49.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.086+0000 7f0470eac700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f044c066f30 0x7f044c0693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:49.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.086+0000 7f0470eac700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f046c0c9f20 0x7f046c15d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:49.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.086+0000 7f0470eac700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f046c0aed20 0x7f046c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:49.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.086+0000 7f0470eac700 1 --2- 172.21.15.7:0/3667476815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c0c9b50 0x7f046c15d3f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:49.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.086+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 >> 172.21.15.7:0/3667476815 conn(0x7f046c0cd1a0 msgr2=0x7f046c1560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:49.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.086+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 shutdown_connections 2023-10-07T07:58:49.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:49.086+0000 7f0470eac700 1 -- 172.21.15.7:0/3667476815 wait complete. 2023-10-07T07:58:49.099 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T07:58:49.100 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T07:58:49.100 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-10-07T07:58:49.100 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-07T07:58:49.100 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:49.100 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T07:58:49.100 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T07:58:49.100 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T07:58:49.100 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T07:58:49.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:49.101 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-10-07T07:58:49.124 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:58:49.124 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:58:49.124 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:58:49.125 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42393sec preferred_lft 42393sec 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 36:e1:63:77:36:7f brd ff:ff:ff:ff:ff:ff 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2:8eff:fee3:e22b/64 scope link 2023-10-07T07:58:49.126 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:49.127 INFO:teuthology.orchestra.run.smithi172.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:58:49.127 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 36:e1:63:77:36:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T07:58:49.127 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::34e1:63ff:fe77:367f/64 scope link 2023-10-07T07:58:49.127 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:49.127 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:58:49.127 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether fe:48:36:09:05:1a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-07T07:58:49.127 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::fc48:36ff:fe09:51a/64 scope link 2023-10-07T07:58:49.127 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:58:49.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:49.127 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T07:58:49.196 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-10-07T07:58:49.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:49.198 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-10-07T07:58:49.260 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-10-07T07:58:49.260 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T07:58:49.260 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T07:58:49.261 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-07T07:58:49.261 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T07:58:49.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:49.261 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:58:49.261 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:58:49.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T07:58:49.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T07:58:49.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T07:58:49.261 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:58:49.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:49.372 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-10-07T07:58:49.390 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-07T07:58:49.390 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-07T07:58:49.390 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T07:58:49.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:58:49.390 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2023-10-07T07:58:49.458 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-07T07:58:49.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:49.458 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:49.516 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-07T07:58:49.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:58:49.516 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:49.574 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2023-10-07T07:58:49.641 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:49.641 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:49.708 INFO:teuthology.orchestra.run.smithi172.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:49.709 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:49.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:49.710 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2023-10-07T07:58:49.768 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T07:58:49.768 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-07T07:58:49.810 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2023-10-07T07:58:49.837 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:58:49.837 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:58:49.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:2023-10-07T07:58:49.891+0000 7ff505ba4540 -1 init, newargv = 0x55a79b037bd0 newargc=15 2023-10-07T07:58:49.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:ceph-fuse[128298]: starting ceph client 2023-10-07T07:58:49.897 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:ceph-fuse[128298]: starting fuse 2023-10-07T07:58:49.906 INFO:teuthology.orchestra.run.smithi172.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:58:49.907 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:58:49.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:49.908 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2023-10-07T07:58:49.966 INFO:teuthology.orchestra.run.smithi172.stdout:46 2023-10-07T07:58:49.966 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-07T07:58:49.966 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> import glob 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> import re 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> import os 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> import subprocess 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> def _find_admin_socket(client_name): 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> files = glob.glob(asok_path) 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> # Given a non-glob path, it better be there 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> if "*" not in asok_path: 2023-10-07T07:58:49.967 DEBUG:teuthology.orchestra.run.smithi172:> assert(len(files) == 1) 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> return files[0] 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> for f in files: 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> contents = proc_f.read() 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> if mountpoint in contents: 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> return f 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> print(_find_admin_socket("client.1")) 2023-10-07T07:58:49.968 DEBUG:teuthology.orchestra.run.smithi172:> ' 2023-10-07T07:58:50.089 INFO:teuthology.orchestra.run.smithi172.stdout:/var/run/ceph/ceph-client.1.128298.asok 2023-10-07T07:58:50.272 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.128298.asok 2023-10-07T07:58:50.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:50.272 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.128298.asok status 2023-10-07T07:58:50.426 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: "metadata": { 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: "entity_id": "1", 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": "smithi172", 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: "pid": "128298", 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: "root": "/" 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:50.427 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_count": 0, 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_pinned_count": 0, 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 5041, 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "inst": { 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "name": { 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "client", 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "num": 5041 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 2298631025 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: } 2023-10-07T07:58:50.428 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 2298631025 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "inst_str": "client.5041 192.168.0.1:0/2298631025", 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "addr_str": "192.168.0.1:0/2298631025", 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "inode_count": 1, 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_epoch": 16, 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch": 33, 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch_barrier": 0, 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "blocklisted": false, 2023-10-07T07:58:50.429 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs" 2023-10-07T07:58:50.430 INFO:teuthology.orchestra.run.smithi172.stdout:} 2023-10-07T07:58:50.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:50.436 DEBUG:teuthology.orchestra.run.smithi172:> stat --file-system '--printf=%T 2023-10-07T07:58:50.437 DEBUG:teuthology.orchestra.run.smithi172:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:50.454 INFO:teuthology.orchestra.run.smithi172.stdout:fuseblk 2023-10-07T07:58:50.455 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:50.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:50.455 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-07T07:58:50.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 1 2023-10-07T07:58:50.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.665+0000 7fa308fc4700 1 Processor -- start 2023-10-07T07:58:50.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.665+0000 7fa308fc4700 1 -- start start 2023-10-07T07:58:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa308fc4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040cbee0 0x7fa3040c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa308fc4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3040c72b0 0x7fa304068000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa308fc4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3040cc2b0 0x7fa304068540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa308fc4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3040c8360 con 0x7fa3040c72b0 2023-10-07T07:58:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa308fc4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3040c7750 con 0x7fa3040cc2b0 2023-10-07T07:58:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa308fc4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040c7a30 con 0x7fa3040cbee0 2023-10-07T07:58:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa301d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3040c72b0 0x7fa304068000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa301d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3040c72b0 0x7fa304068000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51450/0 (socket says 172.21.15.7:51450) 2023-10-07T07:58:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa301d9b700 1 -- 172.21.15.7:0/3687328052 learned_addr learned my addr 172.21.15.7:0/3687328052 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa302d9d700 1 --2- 172.21.15.7:0/3687328052 >> v2:172.21.15.172:3300/0 conn(0x7fa3040cc2b0 0x7fa304068540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa30259c700 1 --2- 172.21.15.7:0/3687328052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040cbee0 0x7fa3040c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa301d9b700 1 -- 172.21.15.7:0/3687328052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040cbee0 msgr2=0x7fa3040c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa301d9b700 1 --2- 172.21.15.7:0/3687328052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040cbee0 0x7fa3040c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa301d9b700 1 -- 172.21.15.7:0/3687328052 >> v2:172.21.15.172:3300/0 conn(0x7fa3040cc2b0 msgr2=0x7fa304068540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa301d9b700 1 --2- 172.21.15.7:0/3687328052 >> v2:172.21.15.172:3300/0 conn(0x7fa3040cc2b0 0x7fa304068540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.666+0000 7fa301d9b700 1 -- 172.21.15.7:0/3687328052 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3040c7d10 con 0x7fa3040c72b0 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.667+0000 7fa301d9b700 1 --2- 172.21.15.7:0/3687328052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040c72b0 0x7fa304068000 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa3040694b0 tx=0x7fa2f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cef9cfefa9a7fc server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.667+0000 7fa30159a700 1 -- 172.21.15.7:0/3687328052 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f800b640 con 0x7fa3040c72b0 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.667+0000 7fa30159a700 1 -- 172.21.15.7:0/3687328052 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f800b7a0 con 0x7fa3040c72b0 2023-10-07T07:58:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.667+0000 7fa30159a700 1 -- 172.21.15.7:0/3687328052 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f800bb60 con 0x7fa3040c72b0 2023-10-07T07:58:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.667+0000 7fa308fc4700 1 -- 172.21.15.7:0/3687328052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040c72b0 msgr2=0x7fa304068000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.667+0000 7fa308fc4700 1 --2- 172.21.15.7:0/3687328052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040c72b0 0x7fa304068000 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa3040694b0 tx=0x7fa2f8009250 comp rx=0 tx=0).stop 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.667+0000 7fa308fc4700 1 -- 172.21.15.7:0/3687328052 shutdown_connections 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.668+0000 7fa308fc4700 1 --2- 172.21.15.7:0/3687328052 >> v2:172.21.15.172:3300/0 conn(0x7fa3040cc2b0 0x7fa304068540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.668+0000 7fa308fc4700 1 --2- 172.21.15.7:0/3687328052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040cbee0 0x7fa3040c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.668+0000 7fa308fc4700 1 --2- 172.21.15.7:0/3687328052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040c72b0 0x7fa304068000 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.668+0000 7fa308fc4700 1 -- 172.21.15.7:0/3687328052 >> 172.21.15.7:0/3687328052 conn(0x7fa3040c9c30 msgr2=0x7fa30406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.668+0000 7fa308fc4700 1 -- 172.21.15.7:0/3687328052 shutdown_connections 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.668+0000 7fa308fc4700 1 -- 172.21.15.7:0/3687328052 wait complete. 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.669+0000 7fa308fc4700 1 Processor -- start 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.669+0000 7fa308fc4700 1 -- start start 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa308fc4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040c72b0 0x7fa30416a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:50.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa308fc4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3040cbee0 0x7fa30414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa308fc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040cc2b0 0x7fa3040b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa308fc4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30414ba00 con 0x7fa3040cc2b0 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa308fc4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040b2390 con 0x7fa3040cbee0 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa308fc4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa304169cb0 con 0x7fa3040c72b0 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa301d9b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3040cbee0 0x7fa30414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa30259c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040c72b0 0x7fa30416a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa301d9b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3040cbee0 0x7fa30414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:34550/0 (socket says 172.21.15.7:34550) 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa301d9b700 1 -- 172.21.15.7:0/1605937558 learned_addr learned my addr 172.21.15.7:0/1605937558 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa302d9d700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040cc2b0 0x7fa3040b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa301d9b700 1 -- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040c72b0 msgr2=0x7fa30416a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa301d9b700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040c72b0 0x7fa30416a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa301d9b700 1 -- 172.21.15.7:0/1605937558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040cc2b0 msgr2=0x7fa3040b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:50.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa301d9b700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040cc2b0 0x7fa3040b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa301d9b700 1 -- 172.21.15.7:0/1605937558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f80094f0 con 0x7fa3040cbee0 2023-10-07T07:58:50.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa302d9d700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040cc2b0 0x7fa3040b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:58:50.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.670+0000 7fa301d9b700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3040cbee0 0x7fa30414b4c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8006fd0 tx=0x7fa2f8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.672+0000 7fa2f27fc700 1 -- 172.21.15.7:0/1605937558 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8000d10 con 0x7fa3040cbee0 2023-10-07T07:58:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.672+0000 7fa2f27fc700 1 -- 172.21.15.7:0/1605937558 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f800bb20 con 0x7fa3040cbee0 2023-10-07T07:58:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.672+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3040b20b0 con 0x7fa3040cbee0 2023-10-07T07:58:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.672+0000 7fa2f27fc700 1 -- 172.21.15.7:0/1605937558 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8011670 con 0x7fa3040cbee0 2023-10-07T07:58:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.672+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30416a230 con 0x7fa3040cbee0 2023-10-07T07:58:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.673+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30416a420 con 0x7fa3040cbee0 2023-10-07T07:58:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.673+0000 7fa2f27fc700 1 -- 172.21.15.7:0/1605937558 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2f80119e0 con 0x7fa3040cbee0 2023-10-07T07:58:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.673+0000 7fa2f27fc700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa2e4066f30 0x7fa2e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.673+0000 7fa2f27fc700 1 -- 172.21.15.7:0/1605937558 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa2f8085d50 con 0x7fa3040cbee0 2023-10-07T07:58:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.674+0000 7fa30259c700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa2e4066f30 0x7fa2e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.674+0000 7fa30259c700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa2e4066f30 0x7fa2e40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3040b0500 tx=0x7fa2f4009120 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.678+0000 7fa2f27fc700 1 -- 172.21.15.7:0/1605937558 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2f8089030 con 0x7fa3040cbee0 2023-10-07T07:58:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.880+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) v1 -- 0x7fa30404c3c0 con 0x7fa3040cbee0 2023-10-07T07:58:50.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.886+0000 7fa2f27fc700 1 -- 172.21.15.7:0/1605937558 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v11)=0 v11) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa2f8052fc0 con 0x7fa3040cbee0 2023-10-07T07:58:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa2e4066f30 msgr2=0x7fa2e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa2e4066f30 0x7fa2e40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3040b0500 tx=0x7fa2f4009120 comp rx=0 tx=0).stop 2023-10-07T07:58:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3040cbee0 msgr2=0x7fa30414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3040cbee0 0x7fa30414b4c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8006fd0 tx=0x7fa2f8017000 comp rx=0 tx=0).stop 2023-10-07T07:58:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 shutdown_connections 2023-10-07T07:58:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fa2e4066f30 0x7fa2e40693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3040cc2b0 0x7fa3040b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3040c72b0 0x7fa30416a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 --2- 172.21.15.7:0/1605937558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3040cbee0 0x7fa30414b4c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 >> 172.21.15.7:0/1605937558 conn(0x7fa3040c9c30 msgr2=0x7fa3040c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 shutdown_connections 2023-10-07T07:58:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:50.888+0000 7fa308fc4700 1 -- 172.21.15.7:0/1605937558 wait complete. 2023-10-07T07:58:50.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 37 2023-10-07T07:58:51.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.038+0000 7f607cf0b700 1 Processor -- start 2023-10-07T07:58:51.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.038+0000 7f607cf0b700 1 -- start start 2023-10-07T07:58:51.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f607cf0b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6078068710 0x7f60780ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:51.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f607cf0b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60780cab00 con 0x7f607815c920 2023-10-07T07:58:51.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f607cf0b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607806a7a0 con 0x7f607815ccf0 2023-10-07T07:58:51.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f607cf0b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60780cb3c0 con 0x7f6078068710 2023-10-07T07:58:51.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f6075d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6078068710 0x7f60780ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:51.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f607659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f607815c920 0x7f6078068000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37126/0 (socket says 172.21.15.7:37126) 2023-10-07T07:58:51.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f607659c700 1 -- 172.21.15.7:0/543581590 learned_addr learned my addr 172.21.15.7:0/543581590 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f6075d9b700 1 --2- 172.21.15.7:0/543581590 >> v2:172.21.15.7:3300/0 conn(0x7f6078068710 0x7f60780ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51466/0 (socket says 172.21.15.7:51466) 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f6075d9b700 1 -- 172.21.15.7:0/543581590 >> v1:172.21.15.172:6789/0 conn(0x7f607815ccf0 legacy=0x7f607806ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f6075d9b700 1 -- 172.21.15.7:0/543581590 >> v1:172.21.15.7:6789/0 conn(0x7f607815c920 legacy=0x7f6078068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f6075d9b700 1 -- 172.21.15.7:0/543581590 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60780cae00 con 0x7f6078068710 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.039+0000 7f6075d9b700 1 --2- 172.21.15.7:0/543581590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 0x7f60780ca520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f606c0098e0 tx=0x7f606c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808d753d8be4abaf server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607559a700 1 -- 172.21.15.7:0/543581590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c009e80 con 0x7f6078068710 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607559a700 1 -- 172.21.15.7:0/543581590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c00b860 con 0x7f6078068710 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607559a700 1 -- 172.21.15.7:0/543581590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c00bbe0 con 0x7f6078068710 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607cf0b700 1 -- 172.21.15.7:0/543581590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 msgr2=0x7f60780ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607cf0b700 1 --2- 172.21.15.7:0/543581590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 0x7f60780ca520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f606c0098e0 tx=0x7f606c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:58:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607cf0b700 1 -- 172.21.15.7:0/543581590 shutdown_connections 2023-10-07T07:58:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607cf0b700 1 --2- 172.21.15.7:0/543581590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 0x7f60780ca520 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607cf0b700 1 -- 172.21.15.7:0/543581590 >> 172.21.15.7:0/543581590 conn(0x7f60780c9c30 msgr2=0x7f607806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607cf0b700 1 -- 172.21.15.7:0/543581590 shutdown_connections 2023-10-07T07:58:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.040+0000 7f607cf0b700 1 -- 172.21.15.7:0/543581590 wait complete. 2023-10-07T07:58:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.042+0000 7f607cf0b700 1 Processor -- start 2023-10-07T07:58:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.042+0000 7f607cf0b700 1 -- start start 2023-10-07T07:58:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.042+0000 7f607cf0b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 0x7f60780c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.042+0000 7f607cf0b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f607815c920 0x7f60780af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.042+0000 7f607cf0b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f607815ccf0 0x7f60780afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.042+0000 7f607cf0b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607815ed50 con 0x7f6078068710 2023-10-07T07:58:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.042+0000 7f607cf0b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780c58d0 con 0x7f607815ccf0 2023-10-07T07:58:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.042+0000 7f607cf0b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60780c5bb0 con 0x7f607815c920 2023-10-07T07:58:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.042+0000 7f607659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 0x7f60780c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 0x7f60780c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51468/0 (socket says 172.21.15.7:51468) 2023-10-07T07:58:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607659c700 1 -- 172.21.15.7:0/1397562074 learned_addr learned my addr 172.21.15.7:0/1397562074 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:58:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f6075d9b700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f607815c920 0x7f60780af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f6076d9d700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f607815ccf0 0x7f60780afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607659c700 1 -- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f607815c920 msgr2=0x7f60780af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607659c700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f607815c920 0x7f60780af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607659c700 1 -- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f607815ccf0 msgr2=0x7f60780afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607659c700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f607815ccf0 0x7f60780afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607659c700 1 -- 172.21.15.7:0/1397562074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c009420 con 0x7f6078068710 2023-10-07T07:58:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f6076d9d700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f607815ccf0 0x7f60780afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:58:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607659c700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 0x7f60780c4e30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f606800c7c0 tx=0x7f6068007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f60667fc700 1 -- 172.21.15.7:0/1397562074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60680073d0 con 0x7f6078068710 2023-10-07T07:58:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f60667fc700 1 -- 172.21.15.7:0/1397562074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6068007530 con 0x7f6078068710 2023-10-07T07:58:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f60667fc700 1 -- 172.21.15.7:0/1397562074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068005a40 con 0x7f6078068710 2023-10-07T07:58:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607815e700 con 0x7f6078068710 2023-10-07T07:58:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.043+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607814b390 con 0x7f6078068710 2023-10-07T07:58:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.044+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607804c3c0 con 0x7f6078068710 2023-10-07T07:58:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.045+0000 7f60667fc700 1 -- 172.21.15.7:0/1397562074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f606801b400 con 0x7f6078068710 2023-10-07T07:58:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.045+0000 7f60667fc700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:58:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.045+0000 7f60667fc700 1 -- 172.21.15.7:0/1397562074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6068083fa0 con 0x7f6078068710 2023-10-07T07:58:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.045+0000 7f6075d9b700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:58:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.046+0000 7f6075d9b700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6058066f30 0x7f60580693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6078068b10 tx=0x7f606c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:58:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.050+0000 7f60667fc700 1 -- 172.21.15.7:0/1397562074 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6068051290 con 0x7f6078068710 2023-10-07T07:58:51.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.254+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) v1 -- 0x7f6078069540 con 0x7f6078068710 2023-10-07T07:58:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.258+0000 7f60667fc700 1 -- 172.21.15.7:0/1397562074 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v12)=0 v12) v1 ==== 135+0+0 (secure 0 0 0) 0x7f60680548b0 con 0x7f6078068710 2023-10-07T07:58:51.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.260+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6058066f30 msgr2=0x7f60580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.260+0000 7f607cf0b700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6058066f30 0x7f60580693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6078068b10 tx=0x7f606c017000 comp rx=0 tx=0).stop 2023-10-07T07:58:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.260+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 msgr2=0x7f60780c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:58:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.261+0000 7f607cf0b700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 0x7f60780c4e30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f606800c7c0 tx=0x7f6068007de0 comp rx=0 tx=0).stop 2023-10-07T07:58:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.261+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 shutdown_connections 2023-10-07T07:58:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.261+0000 7f607cf0b700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.261+0000 7f607cf0b700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f607815ccf0 0x7f60780afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.261+0000 7f607cf0b700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6078068710 0x7f60780c4e30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:51.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.261+0000 7f607cf0b700 1 --2- 172.21.15.7:0/1397562074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f607815c920 0x7f60780af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:58:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.261+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 >> 172.21.15.7:0/1397562074 conn(0x7f60780c9c30 msgr2=0x7f607806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:58:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.261+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 shutdown_connections 2023-10-07T07:58:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:58:51.261+0000 7f607cf0b700 1 -- 172.21.15.7:0/1397562074 wait complete. 2023-10-07T07:58:51.275 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> n = 25 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2023-10-07T07:58:51.276 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> try: 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2023-10-07T07:58:51.277 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:51.278 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T07:58:51.278 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2023-10-07T07:58:51.278 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2023-10-07T07:58:51.278 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T07:58:51.278 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:51.278 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T07:58:51.278 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2023-10-07T07:58:51.278 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2023-10-07T07:58:51.278 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T07:58:51.541 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> n = 25 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.542 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> try: 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2023-10-07T07:58:51.543 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2023-10-07T07:58:51.544 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2023-10-07T07:58:51.545 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T07:58:51.853 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> n = 25 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2023-10-07T07:58:51.854 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> try: 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2023-10-07T07:58:51.855 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2023-10-07T07:58:51.856 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:51.856 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2023-10-07T07:58:51.856 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:51.856 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T07:58:51.856 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2023-10-07T07:58:51.856 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2023-10-07T07:58:51.856 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T07:58:51.856 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:51.856 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T07:58:51.857 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2023-10-07T07:58:51.857 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2023-10-07T07:58:51.857 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T07:58:52.165 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:52.165 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T07:58:52.165 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2023-10-07T07:58:52.165 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:52.165 DEBUG:teuthology.orchestra.run.smithi007:> n = 25 2023-10-07T07:58:52.165 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2023-10-07T07:58:52.165 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> try: 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2023-10-07T07:58:52.166 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2023-10-07T07:58:52.167 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T07:58:52.168 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T07:58:52.168 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T07:58:52.168 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2023-10-07T07:58:52.168 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2023-10-07T07:58:52.168 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T07:58:52.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T07:58:52.476 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-07T07:58:52.477 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-07T07:58:52.478 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-07T07:58:52.478 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T07:58:52.478 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:58:52.478 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2023-10-07T07:58:52.478 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T07:58:52.619 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.133382.asok 2023-10-07T07:58:52.800 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.133382.asok 2023-10-07T07:58:52.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:52.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.133382.asok mds_sessions 2023-10-07T07:58:52.975 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:58:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5142, 2023-10-07T07:58:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-07T07:58:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-07T07:58:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-07T07:58:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 5142 2023-10-07T07:58:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T07:58:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 377970135 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.5142 v1:192.168.0.1:0/377970135", 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "v1:192.168.0.1:0/377970135", 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions": [ 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": 0, 2023-10-07T07:58:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addrs": { 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6832", 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 786560894 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6834", 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 786560894 2023-10-07T07:58:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "seq": 204, 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_gen": 0, 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_ttl": "2023-10-07T07:59:48.971581+0000", 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "last_cap_renew_request": "2023-10-07T07:58:48.971581+0000", 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_renew_seq": 2, 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "num_caps": 100, 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "open" 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2023-10-07T07:58:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 16 2023-10-07T07:58:52.979 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:58:52.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:58:52.986 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'find . | wc') 2023-10-07T07:58:57.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:58:57.843 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:59:27.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:59:27.933 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:59:32.060 INFO:teuthology.orchestra.run.smithi007.stdout: 105 105 1490 2023-10-07T07:59:32.061 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:59:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.200+0000 7fcc68b80700 1 Processor -- start 2023-10-07T07:59:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.200+0000 7fcc68b80700 1 -- start start 2023-10-07T07:59:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc68b80700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc641570a0 0x7fcc640c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc68b80700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6406bd30 0x7fcc640c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc68b80700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc640c3b80 con 0x7fcc64157470 2023-10-07T07:59:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc68b80700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640c2f70 con 0x7fcc641570a0 2023-10-07T07:59:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc68b80700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640c3250 con 0x7fcc6406bd30 2023-10-07T07:59:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc6259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc641570a0 0x7fcc640c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc5bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcc64157470 0x7fcc6406b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44068/0 (socket says 172.21.15.7:44068) 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc5bfff700 1 -- 172.21.15.7:0/1464748351 learned_addr learned my addr 172.21.15.7:0/1464748351 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc62d9d700 1 --2- 172.21.15.7:0/1464748351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6406bd30 0x7fcc640c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.201+0000 7fcc61d9b700 1 -- 172.21.15.7:0/1464748351 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289127414 0 0) 0x7fcc640c3b80 con 0x7fcc64157470 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.202+0000 7fcc61d9b700 1 -- 172.21.15.7:0/1464748351 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc50003570 con 0x7fcc64157470 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.202+0000 7fcc6259c700 1 -- 172.21.15.7:0/1464748351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6406bd30 msgr2=0x7fcc640c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.202+0000 7fcc6259c700 1 --2- 172.21.15.7:0/1464748351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6406bd30 0x7fcc640c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.202+0000 7fcc6259c700 1 -- 172.21.15.7:0/1464748351 >> v1:172.21.15.7:6789/0 conn(0x7fcc64157470 legacy=0x7fcc6406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.202+0000 7fcc6259c700 1 -- 172.21.15.7:0/1464748351 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640c3530 con 0x7fcc641570a0 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.202+0000 7fcc6259c700 1 --2- 172.21.15.7:0/1464748351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 0x7fcc640c8800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcc540098e0 tx=0x7fcc540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56b1af89fc5ac33 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.202+0000 7fcc61d9b700 1 -- 172.21.15.7:0/1464748351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54011630 con 0x7fcc641570a0 2023-10-07T07:59:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.202+0000 7fcc61d9b700 1 -- 172.21.15.7:0/1464748351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc54011790 con 0x7fcc641570a0 2023-10-07T07:59:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.202+0000 7fcc61d9b700 1 -- 172.21.15.7:0/1464748351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54011b30 con 0x7fcc641570a0 2023-10-07T07:59:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.203+0000 7fcc68b80700 1 -- 172.21.15.7:0/1464748351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 msgr2=0x7fcc640c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.203+0000 7fcc68b80700 1 --2- 172.21.15.7:0/1464748351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 0x7fcc640c8800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcc540098e0 tx=0x7fcc540091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.203+0000 7fcc68b80700 1 -- 172.21.15.7:0/1464748351 shutdown_connections 2023-10-07T07:59:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.203+0000 7fcc68b80700 1 --2- 172.21.15.7:0/1464748351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc6406bd30 0x7fcc640c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.203+0000 7fcc68b80700 1 --2- 172.21.15.7:0/1464748351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 0x7fcc640c8800 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.203+0000 7fcc68b80700 1 -- 172.21.15.7:0/1464748351 >> 172.21.15.7:0/1464748351 conn(0x7fcc64067ac0 msgr2=0x7fcc640ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.203+0000 7fcc68b80700 1 -- 172.21.15.7:0/1464748351 shutdown_connections 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.203+0000 7fcc68b80700 1 -- 172.21.15.7:0/1464748351 wait complete. 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.204+0000 7fcc68b80700 1 Processor -- start 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.204+0000 7fcc68b80700 1 -- start start 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc68b80700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6406bd30 0x7fcc64152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc68b80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 0x7fcc6413daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc68b80700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc64157470 0x7fcc6413dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc68b80700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64068100 con 0x7fcc641570a0 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc68b80700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641401c0 con 0x7fcc6406bd30 2023-10-07T07:59:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc68b80700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641404a0 con 0x7fcc64157470 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc5bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 0x7fcc6413daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc5bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 0x7fcc6413daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36940/0 (socket says 172.21.15.7:36940) 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc5bfff700 1 -- 172.21.15.7:0/2439424178 learned_addr learned my addr 172.21.15.7:0/2439424178 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc6259c700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6406bd30 0x7fcc64152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc62d9d700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc64157470 0x7fcc6413dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc5bfff700 1 -- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc64157470 msgr2=0x7fcc6413dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc5bfff700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc64157470 0x7fcc6413dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc5bfff700 1 -- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6406bd30 msgr2=0x7fcc64152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc5bfff700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6406bd30 0x7fcc64152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc5bfff700 1 -- 172.21.15.7:0/2439424178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54009420 con 0x7fcc641570a0 2023-10-07T07:59:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.205+0000 7fcc5bfff700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 0x7fcc6413daa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00c590 tx=0x7fcc4c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.206+0000 7fcc5a7fc700 1 -- 172.21.15.7:0/2439424178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c005b30 con 0x7fcc641570a0 2023-10-07T07:59:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.206+0000 7fcc5a7fc700 1 -- 172.21.15.7:0/2439424178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c005cb0 con 0x7fcc641570a0 2023-10-07T07:59:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.206+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc64140780 con 0x7fcc641570a0 2023-10-07T07:59:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.206+0000 7fcc5a7fc700 1 -- 172.21.15.7:0/2439424178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c007320 con 0x7fcc641570a0 2023-10-07T07:59:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.206+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc64152cc0 con 0x7fcc641570a0 2023-10-07T07:59:32.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.207+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64068590 con 0x7fcc641570a0 2023-10-07T07:59:32.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.207+0000 7fcc5a7fc700 1 -- 172.21.15.7:0/2439424178 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc4c01b3b0 con 0x7fcc641570a0 2023-10-07T07:59:32.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.207+0000 7fcc5a7fc700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc44066f30 0x7fcc440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.208+0000 7fcc5a7fc700 1 -- 172.21.15.7:0/2439424178 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcc4c083960 con 0x7fcc641570a0 2023-10-07T07:59:32.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.208+0000 7fcc6259c700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc44066f30 0x7fcc440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.208+0000 7fcc6259c700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc44066f30 0x7fcc440693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcc540091e0 tx=0x7fcc54017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:32.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.212+0000 7fcc5a7fc700 1 -- 172.21.15.7:0/2439424178 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc64068590 con 0x7fcc641570a0 2023-10-07T07:59:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.464+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc64068590 con 0x7fcc641570a0 2023-10-07T07:59:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.464+0000 7fcc5a7fc700 1 -- 172.21.15.7:0/2439424178 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fcc64068590 con 0x7fcc641570a0 2023-10-07T07:59:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:32.466 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/580346028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":580346028},{"type":"v1","addr":"172.21.15.172:6837","nonce":580346028}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3012863722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3012863722},{"type":"v1","addr":"172.21.15.7:6835","nonce":3012863722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4947,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3733075096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3733075096},{"type":"v1","addr":"172.21.15.172:6836","nonce":3733075096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:58:40.003670+0000","modified":"2023-10-07T07:58:43.589885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4959":{"gid":4959,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/786560894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":786560894},{"type":"v1","addr":"172.21.15.7:6834","nonce":786560894}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T07:59:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc44066f30 msgr2=0x7fcc440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc44066f30 0x7fcc440693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcc540091e0 tx=0x7fcc54017000 comp rx=0 tx=0).stop 2023-10-07T07:59:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 msgr2=0x7fcc6413daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 0x7fcc6413daa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00c590 tx=0x7fcc4c007e00 comp rx=0 tx=0).stop 2023-10-07T07:59:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 shutdown_connections 2023-10-07T07:59:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcc44066f30 0x7fcc440693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc64157470 0x7fcc6413dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6406bd30 0x7fcc64152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 --2- 172.21.15.7:0/2439424178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc641570a0 0x7fcc6413daa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 >> 172.21.15.7:0/2439424178 conn(0x7fcc64067ac0 msgr2=0x7fcc6413ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 shutdown_connections 2023-10-07T07:59:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.467+0000 7fcc68b80700 1 -- 172.21.15.7:0/2439424178 wait complete. 2023-10-07T07:59:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-10-07T07:59:32.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2023-10-07T07:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":553,"msgr_send_messages":520,"msgr_recv_bytes":252194,"msgr_send_bytes":261521,"msgr_created_connections":8,"msgr_active_connections":4,"msgr_running_total_time":0.043973054,"msgr_running_send_time":0.018398376,"msgr_running_recv_time":0.037787049,"msgr_running_fast_dispatch_time":0.002460410,"msgr_send_messages_queue_lat":{"avgcount":520,"sum":0.044562201,"avgtime":0.000085696},"msgr_handle_ack_lat":{"avgcount":931,"sum":0.000180423,"avgtime":0.000000193}},"AsyncMessenger::Worker-1":{"msgr_recv_messages":71,"msgr_send_messages":42,"msgr_recv_bytes":25522,"msgr_send_bytes":118691,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.011178485,"msgr_running_send_time":0.002490953,"msgr_running_recv_time":0.005180204,"msgr_running_fast_dispatch_time":0.003275597,"msgr_send_messages_queue_lat":{"avgcount":41,"sum":0.062307108,"avgtime":0.001519685},"msgr_handle_ack_lat":{"avgcount":56,"sum":0.000025739,"avgtime":0.000000459}},"AsyncMessenger::Worker-2":{"msgr_recv_messages":132,"msgr_send_messages":134,"msgr_recv_bytes":33044,"msgr_send_bytes":611781,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.012667842,"msgr_running_send_time":0.003684103,"msgr_running_recv_time":0.004500642,"msgr_running_fast_dispatch_time":0.003575861,"msgr_send_messages_queue_lat":{"avgcount":132,"sum":0.112974192,"avgtime":0.000855865},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":257,"sum":0.081796033,"avgtime":0.000318272}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000008473,"avgtime":0.000004236}},"mds":{"request":130,"reply":130,"reply_latency":{"avgcount":130,"sum":39.090659927,"avgtime":0.300697384},"slow_reply":0,"forward":0,"dir_fetch":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":114,"inodes_top":0,"inodes_bottom":114,"inodes_pin_tail":0,"inodes_pinned":114,"inodes_expired":0,"inodes_with_caps":100,"caps":101,"subtrees":2,"traverse":155,"traverse_hit":144,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":600,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":100,"root_rbytes":160,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":9,"ceph_cap_op_grant":103,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":199,"handle_client_caps_dirty":100,"handle_client_cap_release":2,"process_request_cap_release":105},"mds_cache":{"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":307,"evex":0,"evtrm":0,"ev":307,"evexg":0,"evexd":0,"segadd":1,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4779077,"rdpos":0,"jlat":{"avgcount":110,"sum":0.237281387,"avgtime":0.002157103},"replayed":0},"mds_mem":{"ino":117,"ino+":117,"ino-":0,"dir":16,"dir+":16,"dir-":0,"dn":114,"dn+":114,"dn-":0,"cap":101,"cap+":106,"cap-":5,"rss":49440,"heap":207108},"mds_server":{"dispatch_client_request":217,"dispatch_server_request":0,"handle_client_request":130,"handle_client_session":110,"handle_peer_request":0,"req_create_latency":{"avgcount":100,"sum":0.061768478,"avgtime":0.000617684},"req_getattr_latency":{"avgcount":17,"sum":0.003797778,"avgtime":0.000223398},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":3,"sum":0.000370802,"avgtime":0.000123600},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":4,"sum":0.005285535,"avgtime":0.001321383},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":5,"sum":39.018990810,"avgtime":7.803798162},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000446524,"avgtime":0.000446524},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":78,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":82,"average_load":41,"avg_session_uptime":42,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_Buffer_bytes":0,"bluestore_Buffer_items":0,"bluestore_Extent_bytes":0,"bluestore_Extent_items":0,"bluestore_Blob_bytes":0,"bluestore_Blob_items":0,"bluestore_SharedBlob_bytes":0,"bluestore_SharedBlob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":213925,"buffer_anon_items":84,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5240,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":376608,"mds_co_items":2318,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":152,"op_send_bytes":603784,"op_resend":37,"op_reply":152,"oplen_avg":{"avgcount":152,"sum":180},"op":152,"op_r":0,"op_w":152,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":110,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":33,"map_full":0,"map_inc":1,"osd_sessions":6,"osd_session_open":6,"osd_session_close":0,"osd_laggy":0,"omap_wr":20,"omap_rd":0,"omap_del":3},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":751,"get_sum":238197,"get_or_fail_fail":0,"get_or_fail_success":751,"take":0,"take_sum":0,"put":751,"put_sum":238197,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":152,"take_sum":603784,"put":152,"put_sum":603784,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":152,"take_sum":152,"put":152,"put_sum":152,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x563f2f9ede00":{"val":0,"max":3758096384,"get_started":0,"get":307,"get_sum":584773,"get_or_fail_fail":0,"get_or_fail_success":307,"take":0,"take_sum":0,"put":110,"put_sum":584773,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-10-07T07:59:32.634 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 4, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 931, "avgtime": 1.93e-07, "sum": 0.000180423 }, "msgr_recv_bytes": 252194, "msgr_recv_messages": 553, "msgr_running_fast_dispatch_time": 0.00246041, "msgr_running_recv_time": 0.037787049, "msgr_running_send_time": 0.018398376, "msgr_running_total_time": 0.043973054, "msgr_send_bytes": 261521, "msgr_send_messages": 520, "msgr_send_messages_queue_lat": { "avgcount": 520, "avgtime": 8.5696e-05, "sum": 0.044562201 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 56, "avgtime": 4.59e-07, "sum": 2.5739e-05 }, "msgr_recv_bytes": 25522, "msgr_recv_messages": 71, "msgr_running_fast_dispatch_time": 0.003275597, "msgr_running_recv_time": 0.005180204, "msgr_running_send_time": 0.002490953, "msgr_running_total_time": 0.011178485, "msgr_send_bytes": 118691, "msgr_send_messages": 42, "msgr_send_messages_queue_lat": { "avgcount": 41, "avgtime": 0.001519685, "sum": 0.062307108 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 33044, "msgr_recv_messages": 132, "msgr_running_fast_dispatch_time": 0.003575861, "msgr_running_recv_time": 0.004500642, "msgr_running_send_time": 0.003684103, "msgr_running_total_time": 0.012667842, "msgr_send_bytes": 611781, "msgr_send_messages": 134, "msgr_send_messages_queue_lat": { "avgcount": 132, "avgtime": 0.000855865, "sum": 0.112974192 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 257, "avgtime": 0.000318272, "sum": 0.081796033 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 4.236e-06, "sum": 8.473e-06 }, "queue_len": 0 }, "mds": { "caps": 101, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 103, "ceph_cap_op_revoke": 9, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 2, "handle_client_caps": 199, "handle_client_caps_dirty": 100, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 114, "inodes_bottom": 114, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 114, "inodes_top": 0, "inodes_with_caps": 100, "load_cent": 600, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 105, "q": 0, "reply": 130, "reply_latency": { "avgcount": 130, "avgtime": 0.300697384, "sum": 39.090659927 }, "request": 130, "root_rbytes": 160, "root_rfiles": 100, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 155, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 144, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 307, "evadd": 307, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 110, "avgtime": 0.002157103, "sum": 0.237281387 }, "rdpos": 0, "replayed": 0, "seg": 1, "segadd": 1, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4779077 }, "mds_mem": { "cap": 101, "cap+": 106, "cap-": 5, "dir": 16, "dir+": 16, "dir-": 0, "dn": 114, "dn+": 114, "dn-": 0, "heap": 207108, "ino": 117, "ino+": 117, "ino-": 0, "rss": 49440 }, "mds_server": { "cap_acquisition_throttle": 78, "cap_revoke_eviction": 0, "dispatch_client_request": 217, "dispatch_server_request": 0, "handle_client_request": 130, "handle_client_session": 110, "handle_peer_request": 0, "req_create_latency": { "avgcount": 100, "avgtime": 0.000617684, "sum": 0.061768478 }, "req_getattr_latency": { "avgcount": 17, "avgtime": 0.000223398, "sum": 0.003797778 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 3, "avgtime": 0.0001236, "sum": 0.000370802 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 4, "avgtime": 0.001321383, "sum": 0.005285535 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 5, "avgtime": 7.803798162, "sum": 39.01899081 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000446524, "sum": 0.000446524 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 41, "avg_session_uptime": 42, "mdthresh_evicted": 0, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 82 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_Blob_bytes": 0, "bluestore_Blob_items": 0, "bluestore_Buffer_bytes": 0, "bluestore_Buffer_items": 0, "bluestore_Extent_bytes": 0, "bluestore_Extent_items": 0, "bluestore_SharedBlob_bytes": 0, "bluestore_SharedBlob_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 213925, "buffer_anon_items": 84, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 376608, "mds_co_items": 2318, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5240, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 33, "map_full": 0, "map_inc": 1, "omap_del": 3, "omap_rd": 0, "omap_wr": 20, "op": 152, "op_active": 0, "op_laggy": 0, "op_pg": 0, "op_r": 0, "op_reply": 152, "op_resend": 37, "op_rmw": 0, "op_send": 152, "op_send_bytes": 603784, "op_w": 152, "oplen_avg": { "avgcount": 152, "sum": 180 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 6, "osd_sessions": 6, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 110, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 751, "get_or_fail_fail": 0, "get_or_fail_success": 751, "get_started": 0, "get_sum": 238197, "max": 104857600, "put": 751, "put_sum": 238197, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 152, "put_sum": 603784, "take": 152, "take_sum": 603784, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 152, "put_sum": 152, "take": 152, "take_sum": 152, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x563f2f9ede00": { "get": 307, "get_or_fail_fail": 0, "get_or_fail_success": 307, "get_started": 0, "get_sum": 584773, "max": 3758096384, "put": 110, "put_sum": 584773, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-10-07T07:59:32.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:59:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.766+0000 7f38ab494700 1 Processor -- start 2023-10-07T07:59:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.766+0000 7f38ab494700 1 -- start start 2023-10-07T07:59:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38ab494700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f38a41570a0 0x7f38a40c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38ab494700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a4157470 0x7f38a406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38ab494700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38a406bb60 0x7f38a406cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38ab494700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a406bfe0 con 0x7f38a406bb60 2023-10-07T07:59:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38ab494700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40c3b80 con 0x7f38a41570a0 2023-10-07T07:59:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38ab494700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40c2f70 con 0x7f38a4157470 2023-10-07T07:59:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9a31700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38a406bb60 0x7f38a406cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9a31700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38a406bb60 0x7f38a406cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36956/0 (socket says 172.21.15.7:36956) 2023-10-07T07:59:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9a31700 1 -- 172.21.15.7:0/785243735 learned_addr learned my addr 172.21.15.7:0/785243735 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9230700 1 --2- 172.21.15.7:0/785243735 >> v2:172.21.15.172:3300/0 conn(0x7f38a41570a0 0x7f38a40c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a8a2f700 1 --2- 172.21.15.7:0/785243735 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a4157470 0x7f38a406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9a31700 1 -- 172.21.15.7:0/785243735 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a4157470 msgr2=0x7f38a406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9a31700 1 --2- 172.21.15.7:0/785243735 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a4157470 0x7f38a406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9a31700 1 -- 172.21.15.7:0/785243735 >> v2:172.21.15.172:3300/0 conn(0x7f38a41570a0 msgr2=0x7f38a40c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9a31700 1 --2- 172.21.15.7:0/785243735 >> v2:172.21.15.172:3300/0 conn(0x7f38a41570a0 0x7f38a40c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9a31700 1 -- 172.21.15.7:0/785243735 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40c3250 con 0x7f38a406bb60 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a8a2f700 1 --2- 172.21.15.7:0/785243735 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a4157470 0x7f38a406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.768+0000 7f38a9a31700 1 --2- 172.21.15.7:0/785243735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a406bb60 0x7f38a406cac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f38a40caa70 tx=0x7f389c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d48c876959feba0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f3897fff700 1 -- 172.21.15.7:0/785243735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00e070 con 0x7f38a406bb60 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f3897fff700 1 -- 172.21.15.7:0/785243735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c00ba50 con 0x7f38a406bb60 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f3897fff700 1 -- 172.21.15.7:0/785243735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0092e0 con 0x7f38a406bb60 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f38ab494700 1 -- 172.21.15.7:0/785243735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a406bb60 msgr2=0x7f38a406cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f38ab494700 1 --2- 172.21.15.7:0/785243735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a406bb60 0x7f38a406cac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f38a40caa70 tx=0x7f389c009840 comp rx=0 tx=0).stop 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f38ab494700 1 -- 172.21.15.7:0/785243735 shutdown_connections 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f38ab494700 1 --2- 172.21.15.7:0/785243735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a406bb60 0x7f38a406cac0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f38ab494700 1 --2- 172.21.15.7:0/785243735 >> v2:172.21.15.172:3300/0 conn(0x7f38a41570a0 0x7f38a40c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f38ab494700 1 --2- 172.21.15.7:0/785243735 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a4157470 0x7f38a406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f38ab494700 1 -- 172.21.15.7:0/785243735 >> 172.21.15.7:0/785243735 conn(0x7f38a4067ac0 msgr2=0x7f38a40ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:32.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f38ab494700 1 -- 172.21.15.7:0/785243735 shutdown_connections 2023-10-07T07:59:32.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.769+0000 7f38ab494700 1 -- 172.21.15.7:0/785243735 wait complete. 2023-10-07T07:59:32.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.771+0000 7f38ab494700 1 Processor -- start 2023-10-07T07:59:32.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.771+0000 7f38ab494700 1 -- start start 2023-10-07T07:59:32.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.772+0000 7f38ab494700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a406bb60 0x7f38a4152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.772+0000 7f38ab494700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a41570a0 0x7f38a4138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.772+0000 7f38ab494700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a4157470 0x7f38a4139320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.772+0000 7f38ab494700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4152e70 con 0x7f38a4157470 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.772+0000 7f38ab494700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4168f80 con 0x7f38a41570a0 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.772+0000 7f38ab494700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4168690 con 0x7f38a406bb60 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.772+0000 7f38a9a31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a4157470 0x7f38a4139320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.772+0000 7f38a9a31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a4157470 0x7f38a4139320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36972/0 (socket says 172.21.15.7:36972) 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.772+0000 7f38a9a31700 1 -- 172.21.15.7:0/4026737242 learned_addr learned my addr 172.21.15.7:0/4026737242 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38a8a2f700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a41570a0 0x7f38a4138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38a9230700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a406bb60 0x7f38a4152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38a9a31700 1 -- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a406bb60 msgr2=0x7f38a4152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38a9a31700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a406bb60 0x7f38a4152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38a9a31700 1 -- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a41570a0 msgr2=0x7f38a4138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:32.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38a9a31700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a41570a0 0x7f38a4138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38a9a31700 1 -- 172.21.15.7:0/4026737242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c009ae0 con 0x7f38a4157470 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38a9a31700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a4157470 0x7f38a4139320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38a40cbc20 tx=0x7f389c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38957fa700 1 -- 172.21.15.7:0/4026737242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0032f0 con 0x7f38a4157470 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38957fa700 1 -- 172.21.15.7:0/4026737242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c008e40 con 0x7f38a4157470 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a4168930 con 0x7f38a4157470 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38957fa700 1 -- 172.21.15.7:0/4026737242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0256b0 con 0x7f38a4157470 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.773+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a4168350 con 0x7f38a4157470 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.774+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a40716c0 con 0x7f38a4157470 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.775+0000 7f38957fa700 1 -- 172.21.15.7:0/4026737242 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f389c003040 con 0x7f38a4157470 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.775+0000 7f38957fa700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3888066f30 0x7f38880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.775+0000 7f38957fa700 1 -- 172.21.15.7:0/4026737242 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f389c085b10 con 0x7f38a4157470 2023-10-07T07:59:32.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.775+0000 7f38a9230700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3888066f30 0x7f38880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:32.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.776+0000 7f38a9230700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3888066f30 0x7f38880693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3898006fd0 tx=0x7f3898008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:32.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:32.780+0000 7f38957fa700 1 -- 172.21.15.7:0/4026737242 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f389c052e30 con 0x7f38a4157470 2023-10-07T07:59:33.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.026+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38a40ca850 con 0x7f38a4157470 2023-10-07T07:59:33.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.027+0000 7f38957fa700 1 -- 172.21.15.7:0/4026737242 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f389c056450 con 0x7f38a4157470 2023-10-07T07:59:33.028 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:33.028 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/580346028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":580346028},{"type":"v1","addr":"172.21.15.172:6837","nonce":580346028}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3012863722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3012863722},{"type":"v1","addr":"172.21.15.7:6835","nonce":3012863722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4947,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3733075096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3733075096},{"type":"v1","addr":"172.21.15.172:6836","nonce":3733075096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:58:40.003670+0000","modified":"2023-10-07T07:58:43.589885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4959":{"gid":4959,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/786560894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":786560894},{"type":"v1","addr":"172.21.15.7:6834","nonce":786560894}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T07:59:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3888066f30 msgr2=0x7f38880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3888066f30 0x7f38880693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3898006fd0 tx=0x7f3898008040 comp rx=0 tx=0).stop 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a4157470 msgr2=0x7f38a4139320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a4157470 0x7f38a4139320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38a40cbc20 tx=0x7f389c018000 comp rx=0 tx=0).stop 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 shutdown_connections 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3888066f30 0x7f38880693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a4157470 0x7f38a4139320 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a406bb60 0x7f38a4152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 --2- 172.21.15.7:0/4026737242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a41570a0 0x7f38a4138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 >> 172.21.15.7:0/4026737242 conn(0x7f38a4067ac0 msgr2=0x7f38a4139d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 shutdown_connections 2023-10-07T07:59:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.029+0000 7f38ab494700 1 -- 172.21.15.7:0/4026737242 wait complete. 2023-10-07T07:59:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-10-07T07:59:33.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-07T07:59:33.197 INFO:teuthology.orchestra.run.smithi007.stdout:[{"id":5041,"entity":{"name":{"type":"client","num":5041},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2298631025}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":43.290352904000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.7206449480940815,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5041 192.168.0.1:0/2298631025","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"08cac0187919fc3a247efdf445f652e2fb30a647","ceph_version":"ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)","entity_id":"1","hostname":"smithi172","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"128298","root":"/"}},{"id":5142,"entity":{"name":{"type":"client","num":5142},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":377970135}},"state":"open","num_leases":0,"num_caps":100,"request_load_avg":80,"uptime":45.213324129,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":3.1279933143644829,"halflife":60},"release_caps":{"value":6.3657788870043426,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":290.39429541179823,"halflife":300},"cap_acquisition":{"value":60.190578456101861,"halflife":30},"last_trim_completed_requests_tid":128,"last_trim_completed_flushes_tid":101,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5142 192.168.0.1:0/377970135","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000025c","length":899}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"08cac0187919fc3a247efdf445f652e2fb30a647","ceph_version":"ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)","entity_id":"0","hostname":"smithi007","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"133382","root":"/"}}] 2023-10-07T07:59:33.197 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi172", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "128298", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2298631025, "type": "any" }, "name": { "num": 5041, "type": "client" } }, "id": 5041, "inst": "client.5041 192.168.0.1:0/2298631025", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.7206449480940815 }, "state": "open", "uptime": 43.290352904 }, { "cap_acquisition": { "halflife": 30, "value": 60.19057845610186 }, "client_metadata": { "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi007", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "133382", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 377970135, "type": "any" }, "name": { "num": 5142, "type": "client" } }, "id": 5142, "inst": "client.5142 192.168.0.1:0/377970135", "last_trim_completed_flushes_tid": 101, "last_trim_completed_requests_tid": 128, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 899, "start": "0x1000000025c" } ], "recall_caps": { "halflife": 60, "value": 3.127993314364483 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 6.365778887004343 }, "request_load_avg": 80, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 290.3942954117982 }, "state": "open", "uptime": 45.213324129 } ] 2023-10-07T07:59:33.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump_historic_ops 2023-10-07T07:59:33.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.332+0000 7f6f275ca700 1 Processor -- start 2023-10-07T07:59:33.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.332+0000 7f6f275ca700 1 -- start start 2023-10-07T07:59:33.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f275ca700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6f200d0250 0x7f6f200d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f275ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200b3b60 0x7f6f200b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f275ca700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f200b61f0 con 0x7f6f20159380 2023-10-07T07:59:33.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f275ca700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f200d0be0 con 0x7f6f200d0250 2023-10-07T07:59:33.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f275ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200b4820 con 0x7f6f200b3b60 2023-10-07T07:59:33.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f24b65700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6f200d0250 0x7f6f200d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f25b67700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200b3b60 0x7f6f200b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f25366700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6f20159380 0x7f6f200b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:52248/0 (socket says 172.21.15.7:52248) 2023-10-07T07:59:33.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f25366700 1 -- 172.21.15.7:0/3874468376 learned_addr learned my addr 172.21.15.7:0/3874468376 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:33.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f25b67700 1 -- 172.21.15.7:0/3874468376 >> v2:172.21.15.172:3300/0 conn(0x7f6f200d0250 msgr2=0x7f6f200d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:33.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f25b67700 1 --2- 172.21.15.7:0/3874468376 >> v2:172.21.15.172:3300/0 conn(0x7f6f200d0250 0x7f6f200d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f25b67700 1 -- 172.21.15.7:0/3874468376 >> v1:172.21.15.172:6789/0 conn(0x7f6f20159380 legacy=0x7f6f200b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f25b67700 1 -- 172.21.15.7:0/3874468376 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200b5000 con 0x7f6f200b3b60 2023-10-07T07:59:33.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.333+0000 7f6f24b65700 1 --2- 172.21.15.7:0/3874468376 >> v2:172.21.15.172:3300/0 conn(0x7f6f200d0250 0x7f6f200d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:59:33.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.334+0000 7f6f25b67700 1 --2- 172.21.15.7:0/3874468376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200b3b60 0x7f6f200b3fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009e60 tx=0x7f6f10009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3a5268d70b50c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:33.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.334+0000 7f6f1ffff700 1 -- 172.21.15.7:0/3874468376 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000e070 con 0x7f6f200b3b60 2023-10-07T07:59:33.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.334+0000 7f6f1ffff700 1 -- 172.21.15.7:0/3874468376 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1000bb60 con 0x7f6f200b3b60 2023-10-07T07:59:33.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.334+0000 7f6f1ffff700 1 -- 172.21.15.7:0/3874468376 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10012430 con 0x7f6f200b3b60 2023-10-07T07:59:33.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.334+0000 7f6f275ca700 1 -- 172.21.15.7:0/3874468376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200b3b60 msgr2=0x7f6f200b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.334+0000 7f6f275ca700 1 --2- 172.21.15.7:0/3874468376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200b3b60 0x7f6f200b3fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009e60 tx=0x7f6f10009780 comp rx=0 tx=0).stop 2023-10-07T07:59:33.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.335+0000 7f6f275ca700 1 -- 172.21.15.7:0/3874468376 shutdown_connections 2023-10-07T07:59:33.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.335+0000 7f6f275ca700 1 --2- 172.21.15.7:0/3874468376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200b3b60 0x7f6f200b3fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.335+0000 7f6f275ca700 1 --2- 172.21.15.7:0/3874468376 >> v2:172.21.15.172:3300/0 conn(0x7f6f200d0250 0x7f6f200d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.335+0000 7f6f275ca700 1 -- 172.21.15.7:0/3874468376 >> 172.21.15.7:0/3874468376 conn(0x7f6f20067be0 msgr2=0x7f6f20067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:33.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.335+0000 7f6f275ca700 1 -- 172.21.15.7:0/3874468376 shutdown_connections 2023-10-07T07:59:33.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.335+0000 7f6f275ca700 1 -- 172.21.15.7:0/3874468376 wait complete. 2023-10-07T07:59:33.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.336+0000 7f6f275ca700 1 Processor -- start 2023-10-07T07:59:33.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.336+0000 7f6f275ca700 1 -- start start 2023-10-07T07:59:33.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.337+0000 7f6f275ca700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f200b3b60 0x7f6f201693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.337+0000 7f6f275ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200d0250 0x7f6f2014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f275ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f20159380 0x7f6f2015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f275ca700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f20169910 con 0x7f6f20159380 2023-10-07T07:59:33.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f275ca700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f20156040 con 0x7f6f200b3b60 2023-10-07T07:59:33.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f275ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f20155750 con 0x7f6f200d0250 2023-10-07T07:59:33.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25b67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f20159380 0x7f6f2015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25b67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f20159380 0x7f6f2015a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36976/0 (socket says 172.21.15.7:36976) 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25b67700 1 -- 172.21.15.7:0/2776073358 learned_addr learned my addr 172.21.15.7:0/2776073358 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f24b65700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200d0250 0x7f6f2014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25366700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f200b3b60 0x7f6f201693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25b67700 1 -- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200d0250 msgr2=0x7f6f2014c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25b67700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200d0250 0x7f6f2014c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25b67700 1 -- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f200b3b60 msgr2=0x7f6f201693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25b67700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f200b3b60 0x7f6f201693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25b67700 1 -- 172.21.15.7:0/2776073358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f14009420 con 0x7f6f20159380 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f25b67700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f20159380 0x7f6f2015a110 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f200b4960 tx=0x7f6f10019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:33.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.338+0000 7f6f1d7fa700 1 -- 172.21.15.7:0/2776073358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10027470 con 0x7f6f20159380 2023-10-07T07:59:33.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.339+0000 7f6f1d7fa700 1 -- 172.21.15.7:0/2776073358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f100275f0 con 0x7f6f20159380 2023-10-07T07:59:33.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.339+0000 7f6f275ca700 1 -- 172.21.15.7:0/2776073358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f10009a20 con 0x7f6f20159380 2023-10-07T07:59:33.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.339+0000 7f6f1d7fa700 1 -- 172.21.15.7:0/2776073358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f100279b0 con 0x7f6f20159380 2023-10-07T07:59:33.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.339+0000 7f6f275ca700 1 -- 172.21.15.7:0/2776073358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f20155d50 con 0x7f6f20159380 2023-10-07T07:59:33.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.340+0000 7f6f1d7fa700 1 -- 172.21.15.7:0/2776073358 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f10024070 con 0x7f6f20159380 2023-10-07T07:59:33.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.341+0000 7f6f1d7fa700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f0c066f30 0x7f6f0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.341+0000 7f6f1d7fa700 1 -- 172.21.15.7:0/2776073358 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f10086e60 con 0x7f6f20159380 2023-10-07T07:59:33.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.341+0000 7f6f25366700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f0c066f30 0x7f6f0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.341+0000 7f6f25366700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f0c066f30 0x7f6f0c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6f20152e30 tx=0x7f6f18007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:33.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.343+0000 7f6f308deb80 1 Processor -- start 2023-10-07T07:59:33.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.343+0000 7f6f308deb80 1 -- start start 2023-10-07T07:59:33.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 0x55a1b87f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b87f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f9bf0 0x55a1b87fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f308deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b86c3340 con 0x55a1b87d6560 2023-10-07T07:59:33.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f308deb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b8700160 con 0x55a1b87f0a70 2023-10-07T07:59:33.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f308deb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1b86fdb20 con 0x55a1b87f9bf0 2023-10-07T07:59:33.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f24b65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b87f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f24b65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b87f96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36982/0 (socket says 172.21.15.7:36982) 2023-10-07T07:59:33.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f24b65700 1 -- 172.21.15.7:0/2583697269 learned_addr learned my addr 172.21.15.7:0/2583697269 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:33.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f25366700 1 --2- 172.21.15.7:0/2583697269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 0x55a1b87f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f25b67700 1 --2- 172.21.15.7:0/2583697269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f9bf0 0x55a1b87fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f24b65700 1 -- 172.21.15.7:0/2583697269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f9bf0 msgr2=0x55a1b87fa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:33.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f24b65700 1 --2- 172.21.15.7:0/2583697269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f9bf0 0x55a1b87fa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f24b65700 1 -- 172.21.15.7:0/2583697269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 msgr2=0x55a1b87f0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f24b65700 1 --2- 172.21.15.7:0/2583697269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 0x55a1b87f0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f24b65700 1 -- 172.21.15.7:0/2583697269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1b85ec0d0 con 0x55a1b87d6560 2023-10-07T07:59:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f24b65700 1 --2- 172.21.15.7:0/2583697269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b87f96b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009de0 tx=0x7f6f140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5279d960317650f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.344+0000 7f6f067fc700 1 -- 172.21.15.7:0/2583697269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14011cf0 con 0x55a1b87d6560 2023-10-07T07:59:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f067fc700 1 -- 172.21.15.7:0/2583697269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f14011e50 con 0x55a1b87d6560 2023-10-07T07:59:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f067fc700 1 -- 172.21.15.7:0/2583697269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14014690 con 0x55a1b87d6560 2023-10-07T07:59:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f308deb80 1 -- 172.21.15.7:0/2583697269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 msgr2=0x55a1b87f96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2583697269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b87f96b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009de0 tx=0x7f6f140091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f308deb80 1 -- 172.21.15.7:0/2583697269 shutdown_connections 2023-10-07T07:59:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2583697269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f9bf0 0x55a1b87fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2583697269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 0x55a1b87f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2583697269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b87f96b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f308deb80 1 -- 172.21.15.7:0/2583697269 >> 172.21.15.7:0/2583697269 conn(0x55a1b87f3ea0 msgr2=0x55a1b87f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f308deb80 1 -- 172.21.15.7:0/2583697269 shutdown_connections 2023-10-07T07:59:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.345+0000 7f6f308deb80 1 -- 172.21.15.7:0/2583697269 wait complete. 2023-10-07T07:59:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.346+0000 7f6f308deb80 1 -- start start 2023-10-07T07:59:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.346+0000 7f6f308deb80 1 Processor -- start 2023-10-07T07:59:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87d6560 0x55a1b87f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 0x55a1b87f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b87efd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f308deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b86f9910 con 0x55a1b87f9bf0 2023-10-07T07:59:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f308deb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b86a59b0 con 0x55a1b87d6560 2023-10-07T07:59:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f308deb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1b86a5650 con 0x55a1b87f0a70 2023-10-07T07:59:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f308deb80 10 client.0 _ll_register_callbacks cb 0x55a1b8508a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T07:59:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f25b67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b87efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f25b67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b87efd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36994/0 (socket says 172.21.15.7:36994) 2023-10-07T07:59:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f25b67700 1 -- 172.21.15.7:0/2400656561 learned_addr learned my addr 172.21.15.7:0/2400656561 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f24b65700 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 0x55a1b87f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.347+0000 7f6f25366700 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87d6560 0x55a1b87f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f25b67700 1 -- 172.21.15.7:0/2400656561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 msgr2=0x55a1b87f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f25b67700 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 0x55a1b87f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f25b67700 1 -- 172.21.15.7:0/2400656561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87d6560 msgr2=0x55a1b87f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f25b67700 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87d6560 0x55a1b87f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f25b67700 1 -- 172.21.15.7:0/2400656561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f14009a90 con 0x55a1b87f9bf0 2023-10-07T07:59:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f25366700 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87d6560 0x55a1b87f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f25b67700 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b87efd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000b920 tx=0x7f6f1000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa63a68cbf0ac614 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6eff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-07T07:59:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2400656561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000e040 con 0x55a1b87f9bf0 2023-10-07T07:59:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2400656561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f1005d0c0 con 0x55a1b87f9bf0 2023-10-07T07:59:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f308deb80 1 -- 172.21.15.7:0/2400656561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1b879e590 con 0x55a1b87f9bf0 2023-10-07T07:59:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2400656561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f1005ed30 con 0x55a1b87f9bf0 2023-10-07T07:59:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f308deb80 10 client.5196 fetch_fsmap learned FSMap version 16 2023-10-07T07:59:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f308deb80 1 -- 172.21.15.7:0/2400656561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55a1b86c83f0 con 0x55a1b87f9bf0 2023-10-07T07:59:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2400656561 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1b879e590 con 0x55a1b87f9bf0 2023-10-07T07:59:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2400656561 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6f10057330 con 0x55a1b87f9bf0 2023-10-07T07:59:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f308deb80 10 client.5196 fetch_fsmap finished waiting for FSMap version 16 2023-10-07T07:59:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f308deb80 10 client.5196 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-10-07T07:59:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f308deb80 20 client.5196 populate_metadata read hostname 'smithi007' 2023-10-07T07:59:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55a1b87abb50 0x55a1b87bdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f308deb80 4 client.5196 mds_command: new command op to 4959 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T07:59:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.348+0000 7f6f308deb80 1 -- 172.21.15.7:0/2400656561 --> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1b877b110 con 0x55a1b87abb50 2023-10-07T07:59:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.349+0000 7f6f25366700 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55a1b87abb50 0x55a1b87bdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.349+0000 7f6eff7fe700 10 client.5196 ms_handle_connect on v2:172.21.15.7:6832/786560894 2023-10-07T07:59:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.349+0000 7f6f25366700 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55a1b87abb50 0x55a1b87bdd10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6293039bba4b5f server_cookie=55719a51d4cfeff2 in_seq=0 out_seq=0 2023-10-07T07:59:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.349+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2400656561 <== mds.0 v2:172.21.15.7:6832/786560894 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a1b877b110 con 0x55a1b87abb50 2023-10-07T07:59:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.349+0000 7f6eff7fe700 10 client.5196 handle_command_reply: tid=0 2023-10-07T07:59:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.349+0000 7f6f308deb80 1 client.5196 shutdown 2023-10-07T07:59:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 -- 172.21.15.7:0/2400656561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 msgr2=0x55a1b87efd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b87efd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000b920 tx=0x7f6f1000bfd0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 -- 172.21.15.7:0/2400656561 shutdown_connections 2023-10-07T07:59:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55a1b87abb50 0x55a1b87bdd10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b87efd50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87d6560 0x55a1b87f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6eff7fe700 0 client.5196 ms_handle_reset on v2:172.21.15.7:6832/786560894 2023-10-07T07:59:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2400656561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 0x55a1b87f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 -- 172.21.15.7:0/2400656561 >> 172.21.15.7:0/2400656561 conn(0x55a1b87f3ea0 msgr2=0x55a1b87a6280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 -- 172.21.15.7:0/2400656561 shutdown_connections 2023-10-07T07:59:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 1 -- 172.21.15.7:0/2400656561 wait complete. 2023-10-07T07:59:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.350+0000 7f6f308deb80 20 client.5196 trim_cache size 0 max 16384 2023-10-07T07:59:33.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.364+0000 7f6f308deb80 1 Processor -- start 2023-10-07T07:59:33.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.365+0000 7f6f308deb80 1 -- start start 2023-10-07T07:59:33.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.365+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87d6560 0x55a1b87aad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.365+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 0x55a1b879f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.365+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b879fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.365+0000 7f6f308deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b8591210 con 0x55a1b87f9bf0 2023-10-07T07:59:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.365+0000 7f6f308deb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b87d62d0 con 0x55a1b87f0a70 2023-10-07T07:59:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.365+0000 7f6f308deb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1b87d4100 con 0x55a1b87d6560 2023-10-07T07:59:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.365+0000 7f6f25b67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b879fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25b67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b879fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37006/0 (socket says 172.21.15.7:37006) 2023-10-07T07:59:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25b67700 1 -- 172.21.15.7:0/4014428327 learned_addr learned my addr 172.21.15.7:0/4014428327 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25366700 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87d6560 0x55a1b87aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f24b65700 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 0x55a1b879f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25b67700 1 -- 172.21.15.7:0/4014428327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87d6560 msgr2=0x55a1b87aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:33.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25b67700 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87d6560 0x55a1b87aad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25b67700 1 -- 172.21.15.7:0/4014428327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 msgr2=0x55a1b879f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:33.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25b67700 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 0x55a1b879f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25b67700 1 -- 172.21.15.7:0/4014428327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1b8560200 con 0x55a1b87f9bf0 2023-10-07T07:59:33.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25366700 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87d6560 0x55a1b87aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f25b67700 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b879fd40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6f10054860 tx=0x7f6f10029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94631a371b121ca6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f06ffd700 1 -- 172.21.15.7:0/4014428327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000e040 con 0x55a1b87f9bf0 2023-10-07T07:59:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f06ffd700 1 -- 172.21.15.7:0/4014428327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f10054180 con 0x55a1b87f9bf0 2023-10-07T07:59:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.366+0000 7f6f06ffd700 1 -- 172.21.15.7:0/4014428327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1005d0c0 con 0x55a1b87f9bf0 2023-10-07T07:59:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.367+0000 7f6f308deb80 1 -- 172.21.15.7:0/4014428327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 msgr2=0x55a1b879fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.367+0000 7f6f308deb80 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b879fd40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6f10054860 tx=0x7f6f10029000 comp rx=0 tx=0).stop 2023-10-07T07:59:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.367+0000 7f6f308deb80 1 -- 172.21.15.7:0/4014428327 shutdown_connections 2023-10-07T07:59:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.367+0000 7f6f308deb80 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87f9bf0 0x55a1b879fd40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.367+0000 7f6f308deb80 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87d6560 0x55a1b87aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.367+0000 7f6f308deb80 1 --2- 172.21.15.7:0/4014428327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f0a70 0x55a1b879f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.367+0000 7f6f308deb80 1 -- 172.21.15.7:0/4014428327 >> 172.21.15.7:0/4014428327 conn(0x55a1b87f3ea0 msgr2=0x55a1b87d59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:33.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.367+0000 7f6f308deb80 1 -- 172.21.15.7:0/4014428327 shutdown_connections 2023-10-07T07:59:33.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.367+0000 7f6f308deb80 1 -- 172.21.15.7:0/4014428327 wait complete. 2023-10-07T07:59:33.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.368+0000 7f6f308deb80 1 -- start start 2023-10-07T07:59:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.368+0000 7f6f308deb80 1 Processor -- start 2023-10-07T07:59:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b884bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 0x55a1b884e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f308deb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f9bf0 0x55a1b884e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f308deb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b8854d70 con 0x55a1b87d6560 2023-10-07T07:59:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f308deb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b87d25f0 con 0x55a1b87f9bf0 2023-10-07T07:59:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f308deb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1b87d27f0 con 0x55a1b87f0a70 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f308deb80 10 client.0 _ll_register_callbacks cb 0x55a1b8508a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f25366700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b884bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f25366700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b884bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37022/0 (socket says 172.21.15.7:37022) 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.369+0000 7f6f25366700 1 -- 172.21.15.7:0/2448227278 learned_addr learned my addr 172.21.15.7:0/2448227278 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f25b67700 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f9bf0 0x55a1b884e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f24b65700 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 0x55a1b884e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f25366700 1 -- 172.21.15.7:0/2448227278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 msgr2=0x55a1b884e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f25366700 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 0x55a1b884e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f25366700 1 -- 172.21.15.7:0/2448227278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f9bf0 msgr2=0x55a1b884e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f25366700 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f9bf0 0x55a1b884e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f25366700 1 -- 172.21.15.7:0/2448227278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1b86c83f0 con 0x55a1b87d6560 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f25b67700 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f9bf0 0x55a1b884e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f25366700 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b884bcc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55a1b87ac110 tx=0x7f6f1800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc289b99cb6bf64 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6eff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2448227278 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18022a60 con 0x55a1b87d6560 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f308deb80 1 -- 172.21.15.7:0/2448227278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f14011e50 con 0x55a1b87d6560 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2448227278 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f1801b040 con 0x55a1b87d6560 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2448227278 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f18022dd0 con 0x55a1b87d6560 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f308deb80 10 client.5202 fetch_fsmap learned FSMap version 16 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6f308deb80 1 -- 172.21.15.7:0/2448227278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55a1b8539d90 con 0x55a1b87d6560 2023-10-07T07:59:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.370+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2448227278 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f14011e50 con 0x55a1b87d6560 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2448227278 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6f1802c450 con 0x55a1b87d6560 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6f308deb80 10 client.5202 fetch_fsmap finished waiting for FSMap version 16 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6f308deb80 10 client.5202 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6f308deb80 20 client.5202 populate_metadata read hostname 'smithi007' 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55a1b87c0980 0x55a1b88582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6f308deb80 4 client.5202 mds_command: new command op to 4959 tid=0[{"prefix": "dump_historic_ops"}] 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6f308deb80 1 -- 172.21.15.7:0/2448227278 --> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] -- command(tid 0: {"prefix": "dump_historic_ops"}) v1 -- 0x55a1b869c900 con 0x55a1b87c0980 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6f24b65700 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55a1b87c0980 0x55a1b88582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6eff7fe700 10 client.5202 ms_handle_connect on v2:172.21.15.7:6832/786560894 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.371+0000 7f6f24b65700 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55a1b87c0980 0x55a1b88582f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617876a8d18c326e server_cookie=a09bf49c3b433b07 in_seq=0 out_seq=0 2023-10-07T07:59:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6eff7fe700 1 -- 172.21.15.7:0/2448227278 <== mds.0 v2:172.21.15.7:6832/786560894 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+68954 (crc 0 0 0) 0x55a1b869c900 con 0x55a1b87c0980 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6eff7fe700 10 client.5202 handle_command_reply: tid=0 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 client.5202 shutdown 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 -- 172.21.15.7:0/2448227278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 msgr2=0x55a1b884bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b884bcc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55a1b87ac110 tx=0x7f6f1800a000 comp rx=0 tx=0).stop 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 -- 172.21.15.7:0/2448227278 shutdown_connections 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a1b87f9bf0 0x55a1b884e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.7:6832/786560894,v1:172.21.15.7:6834/786560894] conn(0x55a1b87c0980 0x55a1b88582f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b87d6560 0x55a1b884bcc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 --2- 172.21.15.7:0/2448227278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a1b87f0a70 0x55a1b884e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6eff7fe700 0 client.5202 ms_handle_reset on v2:172.21.15.7:6832/786560894 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 -- 172.21.15.7:0/2448227278 >> 172.21.15.7:0/2448227278 conn(0x55a1b87f3ea0 msgr2=0x55a1b87d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 -- 172.21.15.7:0/2448227278 shutdown_connections 2023-10-07T07:59:33.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 1 -- 172.21.15.7:0/2448227278 wait complete. 2023-10-07T07:59:33.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.373+0000 7f6f308deb80 20 client.5202 trim_cache size 0 max 16384 2023-10-07T07:59:33.382 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:59:33.382 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2023-10-07T07:59:33.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2023-10-07T07:59:33.382 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2023-10-07T07:59:33.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.382 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:2 setattr mode=041777 #0x1 2023-10-07T07:58:48.640295+0000 caller_uid=0, caller_gid=0{0,})", 2023-10-07T07:59:33.382 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:48.640474+0000", 2023-10-07T07:59:33.382 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.732380001999999, 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2565735179999999, 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:2", 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:48.640474+0000", 2023-10-07T07:59:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:48.640475+0000", 2023-10-07T07:59:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:48.640474+0000", 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:48.640488+0000", 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:48.640546+0000", 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:48.640797+0000", 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:48.640926+0000", 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:48.640928+0000", 2023-10-07T07:59:33.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:49.896854+0000", 2023-10-07T07:59:33.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:49.896918+0000", 2023-10-07T07:59:33.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:49.897003+0000", 2023-10-07T07:59:33.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:49.897046+0000", 2023-10-07T07:59:33.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:49.897047+0000", 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:5 create owner_uid=1000, owner_gid=1285 #0x10000000000/file_0 2023-10-07T07:58:51.377332+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:51.377428+0000", 2023-10-07T07:59:33.389 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.995425797000003, 2023-10-07T07:59:33.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065141599999999997, 2023-10-07T07:59:33.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.390 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:5", 2023-10-07T07:59:33.390 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5 2023-10-07T07:59:33.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.377428+0000", 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.377429+0000", 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.377428+0000", 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.377434+0000", 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.377463+0000", 2023-10-07T07:59:33.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.377695+0000", 2023-10-07T07:59:33.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "failed to rdlock, waiting" 2023-10-07T07:59:33.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.377916+0000", 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.378243+0000", 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.378244+0000", 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.383623+0000", 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.383790+0000", 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.383863+0000", 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.383941+0000", 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.383942+0000", 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:6 create owner_uid=1000, owner_gid=1285 #0x10000000000/file_1 2023-10-07T07:58:51.384568+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.396 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:51.384681+0000", 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.988172327999997, 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067468500000000004, 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:6", 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 6 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.384681+0000", 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.384682+0000", 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.384681+0000", 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.384690+0000", 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.384718+0000", 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.384851+0000", 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.385234+0000", 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.385235+0000", 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.391083+0000", 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.391264+0000", 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.391356+0000", 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.391427+0000", 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.391428+0000", 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:8 create owner_uid=1000, owner_gid=1285 #0x10000000000/file_3 2023-10-07T07:58:51.398743+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:51.398844+0000", 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.974009393999999, 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063180720000000001, 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:8", 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.398844+0000", 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.398845+0000", 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.398844+0000", 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.398851+0000", 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.398882+0000", 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.399012+0000", 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.399430+0000", 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.399431+0000", 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.404710+0000", 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.404990+0000", 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.405078+0000", 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.405161+0000", 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.405162+0000", 2023-10-07T07:59:33.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:9 create owner_uid=1000, owner_gid=1285 #0x10000000000/file_4 2023-10-07T07:58:51.405792+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.408 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:51.405901+0000", 2023-10-07T07:59:33.408 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.966952454999998, 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0089585359999999996, 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:9", 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.405901+0000", 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.405902+0000", 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.405901+0000", 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.405908+0000", 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.405936+0000", 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.406071+0000", 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.406457+0000", 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.406458+0000", 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.414534+0000", 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.414698+0000", 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.414784+0000", 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.414859+0000", 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.414860+0000", 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:11 create owner_uid=1000, owner_gid=1285 #0x10000000000/file_6 2023-10-07T07:58:51.420151+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:51.420242+0000", 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.952611855999997, 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088380739999999996, 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:11", 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.420242+0000", 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.420243+0000", 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.420242+0000", 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.420249+0000", 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.420279+0000", 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.420402+0000", 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.420776+0000", 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.420777+0000", 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.428581+0000", 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.428893+0000", 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.428994+0000", 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.429078+0000", 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.429080+0000", 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:14 create owner_uid=1000, owner_gid=1285 #0x10000000000/file_9 2023-10-07T07:58:51.439590+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:51.439704+0000", 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.933149503999999, 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0062146500000000004, 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:14", 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.439704+0000", 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.439705+0000", 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.439704+0000", 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.439711+0000", 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.439742+0000", 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.439868+0000", 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.440256+0000", 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.440257+0000", 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.445555+0000", 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.445729+0000", 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.445829+0000", 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.445918+0000", 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.445919+0000", 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.425 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:39 create owner_uid=1000, owner_gid=1285 #0x1000000020e/file_6 2023-10-07T07:58:51.715323+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:51.715437+0000", 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.657416560999998, 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061410049999999997, 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:39", 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 39 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.426 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.715437+0000", 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.715438+0000", 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.715437+0000", 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.715445+0000", 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.715474+0000", 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.715598+0000", 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.715966+0000", 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.715967+0000", 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.721070+0000", 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.721377+0000", 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.721487+0000", 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.721576+0000", 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:51.721578+0000", 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:62 create owner_uid=1000, owner_gid=1285 #0x10000000228/file_1 2023-10-07T07:58:52.004035+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.004148+0000", 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.368705898999998, 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063542989999999999, 2023-10-07T07:59:33.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:62", 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 62 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.004148+0000", 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.004149+0000", 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.004148+0000", 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.004154+0000", 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.004177+0000", 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.004314+0000", 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.004668+0000", 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.004668+0000", 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.010214+0000", 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.010379+0000", 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.010444+0000", 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.010502+0000", 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.010502+0000", 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:65 create owner_uid=1000, owner_gid=1285 #0x10000000228/file_4 2023-10-07T07:58:52.019215+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.019305+0000", 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.353549270999999, 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00635696, 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:65", 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 65 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.019305+0000", 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.019305+0000", 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.019305+0000", 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.019311+0000", 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.019335+0000", 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.019482+0000", 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.019895+0000", 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.019895+0000", 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.025273+0000", 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.025486+0000", 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.025585+0000", 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.025660+0000", 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.025661+0000", 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:68 create owner_uid=1000, owner_gid=1285 #0x10000000228/file_7 2023-10-07T07:58:52.037115+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.037212+0000", 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.335641514000002, 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0062817810000000002, 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:68", 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 68 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.037212+0000", 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.037213+0000", 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.037212+0000", 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.037219+0000", 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.037243+0000", 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.037366+0000", 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.037768+0000", 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.037769+0000", 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.043125+0000", 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.043308+0000", 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.043416+0000", 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.043493+0000", 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.043494+0000", 2023-10-07T07:59:33.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:69 create owner_uid=1000, owner_gid=1285 #0x10000000228/file_8 2023-10-07T07:58:52.044201+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.044303+0000", 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.328550929999999, 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060486769999999997, 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.448 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:69", 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 69 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.044303+0000", 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.044304+0000", 2023-10-07T07:59:33.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.044303+0000", 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.044309+0000", 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.044352+0000", 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.044493+0000", 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.044876+0000", 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.044877+0000", 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.049971+0000", 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.050161+0000", 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.050270+0000", 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.050350+0000", 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.050352+0000", 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:70 create owner_uid=1000, owner_gid=1285 #0x10000000228/file_9 2023-10-07T07:58:52.051039+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.051143+0000", 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.321711286000003, 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067322429999999997, 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:70", 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 70 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.051143+0000", 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.051143+0000", 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.051143+0000", 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.051149+0000", 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.051174+0000", 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.051299+0000", 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.051715+0000", 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.051716+0000", 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.057285+0000", 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.057685+0000", 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.057789+0000", 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.057873+0000", 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.057875+0000", 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:71 create owner_uid=1000, owner_gid=1285 #0x10000000228/file_10 2023-10-07T07:58:52.058553+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.459 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.058673+0000", 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.314180710000002, 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060857400000000001, 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:71", 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 71 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.058673+0000", 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.058674+0000", 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.058673+0000", 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.058680+0000", 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.058705+0000", 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.058828+0000", 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.059233+0000", 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.059234+0000", 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.064271+0000", 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.064546+0000", 2023-10-07T07:59:33.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.064658+0000", 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.064757+0000", 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.064759+0000", 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:80 create owner_uid=1000, owner_gid=1285 #0x10000000228/file_19 2023-10-07T07:58:52.104577+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.104695+0000", 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.268158620999998, 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006597299, 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:80", 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 80 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.104695+0000", 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.104696+0000", 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.104695+0000", 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.104704+0000", 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.104736+0000", 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.104877+0000", 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.105311+0000", 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.105311+0000", 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.110578+0000", 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.111064+0000", 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.111192+0000", 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.111290+0000", 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.111292+0000", 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:84 create owner_uid=1000, owner_gid=1285 #0x10000000228/file_23 2023-10-07T07:58:52.125854+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.125970+0000", 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.246884213000001, 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069028409999999998, 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:84", 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 84 2023-10-07T07:59:33.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.125970+0000", 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.125970+0000", 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.125970+0000", 2023-10-07T07:59:33.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.125976+0000", 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.126001+0000", 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.126241+0000", 2023-10-07T07:59:33.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.126926+0000", 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.126927+0000", 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.132359+0000", 2023-10-07T07:59:33.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.132754+0000", 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.132819+0000", 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.132872+0000", 2023-10-07T07:59:33.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.132872+0000", 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:92 create owner_uid=1000, owner_gid=1285 #0x10000000242/file_3 2023-10-07T07:58:52.325203+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.325298+0000", 2023-10-07T07:59:33.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.047555314, 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0062745889999999997, 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:92", 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 92 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.325298+0000", 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.325299+0000", 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.325298+0000", 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.325305+0000", 2023-10-07T07:59:33.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.325328+0000", 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.325451+0000", 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.325839+0000", 2023-10-07T07:59:33.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.325839+0000", 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.331226+0000", 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.331392+0000", 2023-10-07T07:59:33.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.331496+0000", 2023-10-07T07:59:33.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.331572+0000", 2023-10-07T07:59:33.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.331573+0000", 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:110 create owner_uid=1000, owner_gid=1285 #0x10000000242/file_21 2023-10-07T07:58:52.418152+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:52.418251+0000", 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.954603040999999, 2023-10-07T07:59:33.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063572810000000002, 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:110", 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 110 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.418251+0000", 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.418251+0000", 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.418251+0000", 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.418257+0000", 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.418285+0000", 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.418414+0000", 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.418822+0000", 2023-10-07T07:59:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.418822+0000", 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.424316+0000", 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.424490+0000", 2023-10-07T07:59:33.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.424558+0000", 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.424608+0000", 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:52.424608+0000", 2023-10-07T07:59:33.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:126 readdir #0x10000000242 2023-10-07T07:58:53.038263+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:58:53.038352+0000", 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.334501736999997, 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 16.507365704000001, 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:126", 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 126 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:53.038352+0000", 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:53.038353+0000", 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:53.038352+0000", 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:53.038355+0000", 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:53.038378+0000", 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:53.038446+0000", 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:53.538639+0000", 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:54.038817+0000", 2023-10-07T07:59:33.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:54.539015+0000", 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:55.039192+0000", 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:55.539288+0000", 2023-10-07T07:59:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:56.039473+0000", 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:56.539592+0000", 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:57.039759+0000", 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:57.539888+0000", 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:58.040078+0000", 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:58.540202+0000", 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:59.040390+0000", 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:58:59.540582+0000", 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:00.040769+0000", 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:00.540950+0000", 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:01.041124+0000", 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:01.541297+0000", 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:02.041424+0000", 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:02.541625+0000", 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:03.041810+0000", 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:03.542005+0000", 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:04.042158+0000", 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:04.542355+0000", 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:05.042533+0000", 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:05.542712+0000", 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:06.042880+0000", 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:06.543068+0000", 2023-10-07T07:59:33.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:07.043237+0000", 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:07.543425+0000", 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:08.043559+0000", 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:08.543754+0000", 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.043926+0000", 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.544516+0000", 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "failed to rdlock, waiting" 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.545056+0000", 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.545508+0000", 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.545562+0000", 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.545715+0000", 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.545718+0000", 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.5142:129 readdir #0x10000000228 2023-10-07T07:59:09.549301+0000 caller_uid=1000, caller_gid=1285{6,36,1000,1285,})", 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2023-10-07T07:59:09.549393+0000", 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.823460916999998, 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 22.509809388000001, 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": "client.5142:129", 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2023-10-07T07:59:33.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5142", 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 129 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.549393+0000", 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.549394+0000", 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.549393+0000", 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.549397+0000", 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.549429+0000", 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:09.549500+0000", 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:10.049703+0000", 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:10.549906+0000", 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:11.050085+0000", 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:11.550277+0000", 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:12.050467+0000", 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:12.550668+0000", 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:13.050793+0000", 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:13.550974+0000", 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:14.051178+0000", 2023-10-07T07:59:33.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:14.551348+0000", 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:15.051529+0000", 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:15.551703+0000", 2023-10-07T07:59:33.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:16.051882+0000", 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:16.552047+0000", 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:17.052220+0000", 2023-10-07T07:59:33.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:17.552320+0000", 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:18.052509+0000", 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:18.552699+0000", 2023-10-07T07:59:33.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:19.052874+0000", 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:19.553055+0000", 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:20.053245+0000", 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:20.553427+0000", 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:21.053619+0000", 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:21.553804+0000", 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:22.053994+0000", 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:22.554173+0000", 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:23.054361+0000", 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:23.554547+0000", 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:24.054686+0000", 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:24.554881+0000", 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:25.055059+0000", 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:25.555244+0000", 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:26.055386+0000", 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:26.555561+0000", 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:27.055732+0000", 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:27.555916+0000", 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:28.056142+0000", 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:28.556306+0000", 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:29.056472+0000", 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:29.556674+0000", 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:30.056870+0000", 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:30.557033+0000", 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:31.057209+0000", 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:31.557406+0000", 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:32.057815+0000", 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "failed to rdlock, waiting" 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:32.058289+0000", 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:32.058893+0000", 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:32.058969+0000", 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:32.059197+0000", 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2023-10-07T07:59:32.059202+0000", 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:33.525 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:59:33.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 -- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f0c066f30 msgr2=0x7f6f0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f0c066f30 0x7f6f0c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6f20152e30 tx=0x7f6f18007eb0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 -- 172.21.15.7:0/2776073358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f20159380 msgr2=0x7f6f2015a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:33.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f20159380 0x7f6f2015a110 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f200b4960 tx=0x7f6f10019000 comp rx=0 tx=0).stop 2023-10-07T07:59:33.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 -- 172.21.15.7:0/2776073358 shutdown_connections 2023-10-07T07:59:33.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6f0c066f30 0x7f6f0c0693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f20159380 0x7f6f2015a110 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f200b3b60 0x7f6f201693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 --2- 172.21.15.7:0/2776073358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f200d0250 0x7f6f2014c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:33.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 -- 172.21.15.7:0/2776073358 >> 172.21.15.7:0/2776073358 conn(0x7f6f20067be0 msgr2=0x7f6f20153000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:33.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 -- 172.21.15.7:0/2776073358 shutdown_connections 2023-10-07T07:59:33.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:33.374+0000 7f6f275ca700 1 -- 172.21.15.7:0/2776073358 wait complete. 2023-10-07T07:59:33.528 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-07T07:59:33.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T07:59:33.585 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T07:59:33.634 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-07T07:59:33.634 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-07T07:59:33.634 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T07:59:33.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-07T07:59:33.681 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[133382]: fuse finished with error 0 and tester_r 0 2023-10-07T07:59:33.682 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:59:33.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:33.683 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-07T07:59:33.826 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi172.front.sepia.ceph.com... 2023-10-07T07:59:33.826 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T07:59:33.826 DEBUG:teuthology.orchestra.run.smithi172:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-07T07:59:33.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:ceph-fuse[128298]: fuse finished with error 0 and tester_r 0 2023-10-07T07:59:33.864 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:59:33.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:33.865 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-07T07:59:33.946 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T07:59:33.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:33.946 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-07T07:59:33.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T07:59:33.970 INFO:teuthology.orchestra.run.smithi007.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-07T07:59:33.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T07:59:33.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:33.970 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-07T07:59:34.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T07:59:34.006 INFO:teuthology.orchestra.run.smithi172.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-07T07:59:34.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:59:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.149+0000 7f75785ca700 1 Processor -- start 2023-10-07T07:59:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.149+0000 7f75785ca700 1 -- start start 2023-10-07T07:59:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.150+0000 7f75785ca700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f75700ced30 0x7f75700c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.150+0000 7f75785ca700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f75701596e0 0x7f757006b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.150+0000 7f75785ca700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7570068100 con 0x7f7570159310 2023-10-07T07:59:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.150+0000 7f75785ca700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7570068350 con 0x7f75701596e0 2023-10-07T07:59:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.150+0000 7f75785ca700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7570068630 con 0x7f75700ced30 2023-10-07T07:59:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.150+0000 7f7576b67700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f75701596e0 0x7f757006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.150+0000 7f7576b67700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f75701596e0 0x7f757006b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37026/0 (socket says 172.21.15.7:37026) 2023-10-07T07:59:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.150+0000 7f7576b67700 1 -- 172.21.15.7:0/1001719592 learned_addr learned my addr 172.21.15.7:0/1001719592 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.150+0000 7f7575b65700 1 --2- 172.21.15.7:0/1001719592 >> v2:172.21.15.172:3300/0 conn(0x7f75700ced30 0x7f75700c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.151+0000 7f7576b67700 1 -- 172.21.15.7:0/1001719592 >> v2:172.21.15.172:3300/0 conn(0x7f75700ced30 msgr2=0x7f75700c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.151+0000 7f7576b67700 1 --2- 172.21.15.7:0/1001719592 >> v2:172.21.15.172:3300/0 conn(0x7f75700ced30 0x7f75700c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.151+0000 7f7576b67700 1 -- 172.21.15.7:0/1001719592 >> v1:172.21.15.172:6789/0 conn(0x7f7570159310 legacy=0x7f75700c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.151+0000 7f7576b67700 1 -- 172.21.15.7:0/1001719592 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570068910 con 0x7f75701596e0 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.151+0000 7f7576b67700 1 --2- 172.21.15.7:0/1001719592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 0x7f757006b620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f75700cbbf0 tx=0x7f7568009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1201a82a0db3419 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.151+0000 7f7575364700 1 -- 172.21.15.7:0/1001719592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800e070 con 0x7f75701596e0 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.151+0000 7f7575364700 1 -- 172.21.15.7:0/1001719592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800ba20 con 0x7f75701596e0 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.151+0000 7f7575364700 1 -- 172.21.15.7:0/1001719592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75680092e0 con 0x7f75701596e0 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.152+0000 7f75785ca700 1 -- 172.21.15.7:0/1001719592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 msgr2=0x7f757006b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.152+0000 7f75785ca700 1 --2- 172.21.15.7:0/1001719592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 0x7f757006b620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f75700cbbf0 tx=0x7f7568009840 comp rx=0 tx=0).stop 2023-10-07T07:59:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.152+0000 7f75785ca700 1 -- 172.21.15.7:0/1001719592 shutdown_connections 2023-10-07T07:59:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.152+0000 7f75785ca700 1 --2- 172.21.15.7:0/1001719592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 0x7f757006b620 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.152+0000 7f75785ca700 1 --2- 172.21.15.7:0/1001719592 >> v2:172.21.15.172:3300/0 conn(0x7f75700ced30 0x7f75700c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.152+0000 7f75785ca700 1 -- 172.21.15.7:0/1001719592 >> 172.21.15.7:0/1001719592 conn(0x7f7570067ac0 msgr2=0x7f75700ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.152+0000 7f75785ca700 1 -- 172.21.15.7:0/1001719592 shutdown_connections 2023-10-07T07:59:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.152+0000 7f75785ca700 1 -- 172.21.15.7:0/1001719592 wait complete. 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.153+0000 7f75785ca700 1 Processor -- start 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.153+0000 7f75785ca700 1 -- start start 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f75785ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7570159310 0x7f757015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f75785ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 0x7f757015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f75785ca700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75700b1d30 0x7f75701532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f75785ca700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757015a7f0 con 0x7f75701596e0 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f75785ca700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757015aa20 con 0x7f75700b1d30 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f75785ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75700b0f70 con 0x7f7570159310 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7575b65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 0x7f757015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7575b65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 0x7f757015c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37032/0 (socket says 172.21.15.7:37032) 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7575b65700 1 -- 172.21.15.7:0/610337663 learned_addr learned my addr 172.21.15.7:0/610337663 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7576b67700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75700b1d30 0x7f75701532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7576366700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7570159310 0x7f757015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7575b65700 1 -- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7570159310 msgr2=0x7f757015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7575b65700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7570159310 0x7f757015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7575b65700 1 -- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75700b1d30 msgr2=0x7f75701532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7575b65700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75700b1d30 0x7f75701532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.154+0000 7f7575b65700 1 -- 172.21.15.7:0/610337663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7568009ae0 con 0x7f75701596e0 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.155+0000 7f7575b65700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 0x7f757015c290 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f756000e930 tx=0x7f75600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.155+0000 7f75667fc700 1 -- 172.21.15.7:0/610337663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560013680 con 0x7f75701596e0 2023-10-07T07:59:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.155+0000 7f75667fc700 1 -- 172.21.15.7:0/610337663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75600137e0 con 0x7f75701596e0 2023-10-07T07:59:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.155+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75700b1250 con 0x7f75701596e0 2023-10-07T07:59:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.155+0000 7f75667fc700 1 -- 172.21.15.7:0/610337663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560013b80 con 0x7f75701596e0 2023-10-07T07:59:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.155+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75700b1810 con 0x7f75701596e0 2023-10-07T07:59:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.156+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75700716c0 con 0x7f75701596e0 2023-10-07T07:59:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.156+0000 7f75667fc700 1 -- 172.21.15.7:0/610337663 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f756000c3e0 con 0x7f75701596e0 2023-10-07T07:59:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.157+0000 7f75667fc700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7554066f30 0x7f75540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.157+0000 7f75667fc700 1 -- 172.21.15.7:0/610337663 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7560084620 con 0x7f75701596e0 2023-10-07T07:59:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.157+0000 7f7576366700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7554066f30 0x7f75540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.157+0000 7f7576366700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7554066f30 0x7f75540693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f756c003a80 tx=0x7f756c007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:34.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.161+0000 7f75667fc700 1 -- 172.21.15.7:0/610337663 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7560051890 con 0x7f75701596e0 2023-10-07T07:59:34.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.411+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7570158ad0 con 0x7f75701596e0 2023-10-07T07:59:34.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.412+0000 7f75667fc700 1 -- 172.21.15.7:0/610337663 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f7560054eb0 con 0x7f75701596e0 2023-10-07T07:59:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:34.413 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/580346028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":580346028},{"type":"v1","addr":"172.21.15.172:6837","nonce":580346028}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3012863722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3012863722},{"type":"v1","addr":"172.21.15.7:6835","nonce":3012863722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4947,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3733075096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3733075096},{"type":"v1","addr":"172.21.15.172:6836","nonce":3733075096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:58:40.003670+0000","modified":"2023-10-07T07:58:43.589885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4959":{"gid":4959,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/786560894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":786560894},{"type":"v1","addr":"172.21.15.7:6834","nonce":786560894}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T07:59:34.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7554066f30 msgr2=0x7f75540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7554066f30 0x7f75540693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f756c003a80 tx=0x7f756c007ec0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 msgr2=0x7f757015c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 0x7f757015c290 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f756000e930 tx=0x7f75600091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 shutdown_connections 2023-10-07T07:59:34.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7554066f30 0x7f75540693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75700b1d30 0x7f75701532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7570159310 0x7f757015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 --2- 172.21.15.7:0/610337663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75701596e0 0x7f757015c290 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 >> 172.21.15.7:0/610337663 conn(0x7f7570067ac0 msgr2=0x7f757006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:34.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 shutdown_connections 2023-10-07T07:59:34.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.414+0000 7f75785ca700 1 -- 172.21.15.7:0/610337663 wait complete. 2023-10-07T07:59:34.416 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-10-07T07:59:34.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:59:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.560+0000 7f2a8cfcd700 1 Processor -- start 2023-10-07T07:59:34.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.560+0000 7f2a8cfcd700 1 -- start start 2023-10-07T07:59:34.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.561+0000 7f2a8cfcd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a88159690 0x7f2a880cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.561+0000 7f2a8cfcd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a880cb0c0 0x7f2a880a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.561+0000 7f2a8cfcd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8806bb70 con 0x7f2a881592c0 2023-10-07T07:59:34.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.561+0000 7f2a8cfcd700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a880cda10 con 0x7f2a88159690 2023-10-07T07:59:34.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.561+0000 7f2a8cfcd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8806af60 con 0x7f2a880cb0c0 2023-10-07T07:59:34.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2a881592c0 0x7f2a8806a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44072/0 (socket says 172.21.15.7:44072) 2023-10-07T07:59:34.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8659c700 1 -- 172.21.15.7:0/4011990588 learned_addr learned my addr 172.21.15.7:0/4011990588 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:34.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a86d9d700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a880cb0c0 0x7f2a880a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a85d9b700 1 --2- 172.21.15.7:0/4011990588 >> v2:172.21.15.172:3300/0 conn(0x7f2a88159690 0x7f2a880cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328921274 0 0) 0x7f2a8806bb70 con 0x7f2a881592c0 2023-10-07T07:59:34.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a74003570 con 0x7f2a881592c0 2023-10-07T07:59:34.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3006478705 0 0) 0x7f2a74003570 con 0x7f2a881592c0 2023-10-07T07:59:34.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a880cb0c0 msgr2=0x7f2a880a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a880cb0c0 0x7f2a880a9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 >> v2:172.21.15.172:3300/0 conn(0x7f2a88159690 msgr2=0x7f2a880cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 --2- 172.21.15.7:0/4011990588 >> v2:172.21.15.172:3300/0 conn(0x7f2a88159690 0x7f2a880cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8806b240 con 0x7f2a881592c0 2023-10-07T07:59:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f2a78001470 con 0x7f2a881592c0 2023-10-07T07:59:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 >> v1:172.21.15.7:6789/0 conn(0x7f2a881592c0 legacy=0x7f2a8806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a74003a60 0x7f2a74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a880cb0c0 0x7f2a74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74004d50 0x7f2a740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880cda10 con 0x7f2a74003a60 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8806af60 con 0x7f2a74004d50 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7400ba60 con 0x7f2a880cb0c0 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.562+0000 7f2a8659c700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a74003a60 0x7f2a74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a85d9b700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a880cb0c0 0x7f2a74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a86d9d700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74004d50 0x7f2a740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a8659c700 1 -- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a880cb0c0 msgr2=0x7f2a74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a8659c700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a880cb0c0 0x7f2a74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a8659c700 1 -- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74004d50 msgr2=0x7f2a740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:34.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a8659c700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74004d50 0x7f2a740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a8659c700 1 -- 172.21.15.7:0/4011990588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a74003570 con 0x7f2a74003a60 2023-10-07T07:59:34.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a85d9b700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a880cb0c0 0x7f2a74004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:34.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a86d9d700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74004d50 0x7f2a740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:34.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a8659c700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a74003a60 0x7f2a74003e70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c3c0 tx=0x7f2a78001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae11251d1705376 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:34.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800cc30 con 0x7f2a74003a60 2023-10-07T07:59:34.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7800cdb0 con 0x7f2a74003a60 2023-10-07T07:59:34.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.563+0000 7f2a8559a700 1 -- 172.21.15.7:0/4011990588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78005b70 con 0x7f2a74003a60 2023-10-07T07:59:34.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.564+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/4011990588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a74003a60 msgr2=0x7f2a74003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.564+0000 7f2a8cfcd700 1 --2- 172.21.15.7:0/4011990588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a74003a60 0x7f2a74003e70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c3c0 tx=0x7f2a78001e90 comp rx=0 tx=0).stop 2023-10-07T07:59:34.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.564+0000 7f2a86d9d700 1 -- 172.21.15.7:0/4011990588 reap_dead start 2023-10-07T07:59:34.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.564+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/4011990588 shutdown_connections 2023-10-07T07:59:34.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.564+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/4011990588 >> 172.21.15.7:0/4011990588 conn(0x7f2a88068bd0 msgr2=0x7f2a880ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:34.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.564+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/4011990588 shutdown_connections 2023-10-07T07:59:34.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.564+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/4011990588 wait complete. 2023-10-07T07:59:34.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.565+0000 7f2a8cfcd700 1 Processor -- start 2023-10-07T07:59:34.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.565+0000 7f2a8cfcd700 1 -- start start 2023-10-07T07:59:34.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a8cfcd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74003a60 0x7f2a8815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a8cfcd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a8816a1a0 0x7f2a881697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a8cfcd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a8816a6c0 0x7f2a8816ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a8cfcd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815b1f0 con 0x7f2a8816a6c0 2023-10-07T07:59:34.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a8cfcd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815b4c0 con 0x7f2a74003a60 2023-10-07T07:59:34.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a8cfcd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815b7a0 con 0x7f2a8816a1a0 2023-10-07T07:59:34.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a86d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a8816a6c0 0x7f2a8816ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a86d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a8816a6c0 0x7f2a8816ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37060/0 (socket says 172.21.15.7:37060) 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a86d9d700 1 -- 172.21.15.7:0/1240219620 learned_addr learned my addr 172.21.15.7:0/1240219620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a8659c700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74003a60 0x7f2a8815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a85d9b700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a8816a1a0 0x7f2a881697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a86d9d700 1 -- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a8816a1a0 msgr2=0x7f2a881697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a86d9d700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a8816a1a0 0x7f2a881697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a86d9d700 1 -- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74003a60 msgr2=0x7f2a8815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a86d9d700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74003a60 0x7f2a8815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.566+0000 7f2a86d9d700 1 -- 172.21.15.7:0/1240219620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8815ba80 con 0x7f2a8816a6c0 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.567+0000 7f2a86d9d700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a8816a6c0 0x7f2a8816ab40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a88159b30 tx=0x7f2a8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.567+0000 7f2a7e7fc700 1 -- 172.21.15.7:0/1240219620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8000e050 con 0x7f2a8816a6c0 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.567+0000 7f2a7e7fc700 1 -- 172.21.15.7:0/1240219620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8000f040 con 0x7f2a8816a6c0 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.567+0000 7f2a7e7fc700 1 -- 172.21.15.7:0/1240219620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80010d40 con 0x7f2a8816a6c0 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.567+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8815bd60 con 0x7f2a8816a6c0 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.567+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a88169d10 con 0x7f2a8816a6c0 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.568+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8815c280 con 0x7f2a8816a6c0 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.568+0000 7f2a7e7fc700 1 -- 172.21.15.7:0/1240219620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a80003020 con 0x7f2a8816a6c0 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.569+0000 7f2a7e7fc700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2a68066f30 0x7f2a680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.569+0000 7f2a7e7fc700 1 -- 172.21.15.7:0/1240219620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a80084790 con 0x7f2a8816a6c0 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.569+0000 7f2a8659c700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2a68066f30 0x7f2a680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.569+0000 7f2a8659c700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2a68066f30 0x7f2a680693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a78001ed0 tx=0x7f2a78015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:34.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.573+0000 7f2a7e7fc700 1 -- 172.21.15.7:0/1240219620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a8815c280 con 0x7f2a8816a6c0 2023-10-07T07:59:34.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.776+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a8815c280 con 0x7f2a8816a6c0 2023-10-07T07:59:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.777+0000 7f2a7e7fc700 1 -- 172.21.15.7:0/1240219620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2a8815c280 con 0x7f2a8816a6c0 2023-10-07T07:59:34.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:34.779 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:58:41.578137+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:58:37.928486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T07:58:38.977756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:59:34.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2a68066f30 msgr2=0x7f2a680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2a68066f30 0x7f2a680693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a78001ed0 tx=0x7f2a78015000 comp rx=0 tx=0).stop 2023-10-07T07:59:34.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a8816a6c0 msgr2=0x7f2a8816ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a8816a6c0 0x7f2a8816ab40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a88159b30 tx=0x7f2a8000b040 comp rx=0 tx=0).stop 2023-10-07T07:59:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 shutdown_connections 2023-10-07T07:59:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2a68066f30 0x7f2a680693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a8816a6c0 0x7f2a8816ab40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74003a60 0x7f2a8815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 --2- 172.21.15.7:0/1240219620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a8816a1a0 0x7f2a881697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 >> 172.21.15.7:0/1240219620 conn(0x7f2a88068bd0 msgr2=0x7f2a880673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 shutdown_connections 2023-10-07T07:59:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.780+0000 7f2a8cfcd700 1 -- 172.21.15.7:0/1240219620 wait complete. 2023-10-07T07:59:34.795 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T07:58:37.928486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T07:58:38.977756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T07:59:34.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:59:34.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.931+0000 7f2bc55d2700 1 Processor -- start 2023-10-07T07:59:34.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.931+0000 7f2bc55d2700 1 -- start start 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bc55d2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 0x7f2bc00cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bc55d2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bc015dbf0 0x7f2bc00afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bc55d2700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc00cf1e0 con 0x7f2bc00af810 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bc55d2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc00ce5d0 con 0x7f2bc015dbf0 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bc55d2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00ce8b0 con 0x7f2bc00af440 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbe7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bc015dbf0 0x7f2bc00afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbe7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bc015dbf0 0x7f2bc00afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37068/0 (socket says 172.21.15.7:37068) 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbe7fc700 1 -- 172.21.15.7:0/3543885155 learned_addr learned my addr 172.21.15.7:0/3543885155 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbeffd700 1 --2- 172.21.15.7:0/3543885155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 0x7f2bc00cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbe7fc700 1 -- 172.21.15.7:0/3543885155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 msgr2=0x7f2bc00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbe7fc700 1 --2- 172.21.15.7:0/3543885155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 0x7f2bc00cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbe7fc700 1 -- 172.21.15.7:0/3543885155 >> v1:172.21.15.172:6789/0 conn(0x7f2bc00af810 legacy=0x7f2bc00ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbe7fc700 1 -- 172.21.15.7:0/3543885155 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00ceb90 con 0x7f2bc015dbf0 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbe7fc700 1 --2- 172.21.15.7:0/3543885155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 0x7f2bc00afd00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40099b0 tx=0x7f2bb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513b1b1939f4fa25 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbdffb700 1 -- 172.21.15.7:0/3543885155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4009ea0 con 0x7f2bc015dbf0 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbdffb700 1 -- 172.21.15.7:0/3543885155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb400b8f0 con 0x7f2bc015dbf0 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.932+0000 7f2bbdffb700 1 -- 172.21.15.7:0/3543885155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400bc90 con 0x7f2bc015dbf0 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.933+0000 7f2bc55d2700 1 -- 172.21.15.7:0/3543885155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 msgr2=0x7f2bc00afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.933+0000 7f2bc55d2700 1 --2- 172.21.15.7:0/3543885155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 0x7f2bc00afd00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40099b0 tx=0x7f2bb4009250 comp rx=0 tx=0).stop 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.933+0000 7f2bc55d2700 1 -- 172.21.15.7:0/3543885155 shutdown_connections 2023-10-07T07:59:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.933+0000 7f2bc55d2700 1 --2- 172.21.15.7:0/3543885155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 0x7f2bc00cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.933+0000 7f2bc55d2700 1 --2- 172.21.15.7:0/3543885155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 0x7f2bc00afd00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.933+0000 7f2bc55d2700 1 -- 172.21.15.7:0/3543885155 >> 172.21.15.7:0/3543885155 conn(0x7f2bc0069220 msgr2=0x7f2bc0069620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:34.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.933+0000 7f2bc55d2700 1 -- 172.21.15.7:0/3543885155 shutdown_connections 2023-10-07T07:59:34.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.933+0000 7f2bc55d2700 1 -- 172.21.15.7:0/3543885155 wait complete. 2023-10-07T07:59:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.934+0000 7f2bc55d2700 1 Processor -- start 2023-10-07T07:59:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bc55d2700 1 -- start start 2023-10-07T07:59:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bc55d2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 0x7f2bc00c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bc55d2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bc00af810 0x7f2bc00c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bc55d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 0x7f2bc00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bc55d2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00c7cb0 con 0x7f2bc015dbf0 2023-10-07T07:59:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bc55d2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00c7f90 con 0x7f2bc00af810 2023-10-07T07:59:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bc55d2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00c8270 con 0x7f2bc00af440 2023-10-07T07:59:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bbf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 0x7f2bc00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bbf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 0x7f2bc00c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37076/0 (socket says 172.21.15.7:37076) 2023-10-07T07:59:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.935+0000 7f2bbf7fe700 1 -- 172.21.15.7:0/2740801426 learned_addr learned my addr 172.21.15.7:0/2740801426 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bbe7fc700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bc00af810 0x7f2bc00c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bbeffd700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 0x7f2bc00c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bbf7fe700 1 -- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 msgr2=0x7f2bc00c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bbf7fe700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 0x7f2bc00c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bbf7fe700 1 -- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bc00af810 msgr2=0x7f2bc00c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bbf7fe700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bc00af810 0x7f2bc00c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bbf7fe700 1 -- 172.21.15.7:0/2740801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40094f0 con 0x7f2bc015dbf0 2023-10-07T07:59:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bbe7fc700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bc00af810 0x7f2bc00c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bbf7fe700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 0x7f2bc00c63a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c840 tx=0x7f2bb8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2ba6ffd700 1 -- 172.21.15.7:0/2740801426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8012630 con 0x7f2bc015dbf0 2023-10-07T07:59:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2ba6ffd700 1 -- 172.21.15.7:0/2740801426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb80127b0 con 0x7f2bc015dbf0 2023-10-07T07:59:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2ba6ffd700 1 -- 172.21.15.7:0/2740801426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8012b70 con 0x7f2bc015dbf0 2023-10-07T07:59:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc00c8550 con 0x7f2bc015dbf0 2023-10-07T07:59:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.936+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc0152590 con 0x7f2bc015dbf0 2023-10-07T07:59:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.937+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc004c3c0 con 0x7f2bc015dbf0 2023-10-07T07:59:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.938+0000 7f2ba6ffd700 1 -- 172.21.15.7:0/2740801426 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2bb8003030 con 0x7f2bc015dbf0 2023-10-07T07:59:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.938+0000 7f2ba6ffd700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2bac066f60 0x7f2bac069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.938+0000 7f2ba6ffd700 1 -- 172.21.15.7:0/2740801426 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2bb80844f0 con 0x7f2bc015dbf0 2023-10-07T07:59:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.938+0000 7f2bbeffd700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2bac066f60 0x7f2bac069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.939+0000 7f2bbeffd700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2bac066f60 0x7f2bac069410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009ca0 tx=0x7f2bb0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:34.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:34.943+0000 7f2ba6ffd700 1 -- 172.21.15.7:0/2740801426 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bb8051760 con 0x7f2bc015dbf0 2023-10-07T07:59:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.144+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2bc00cdfc0 con 0x7f2bc015dbf0 2023-10-07T07:59:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.144+0000 7f2ba6ffd700 1 -- 172.21.15.7:0/2740801426 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2bb8054d80 con 0x7f2bc015dbf0 2023-10-07T07:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2bac066f60 msgr2=0x7f2bac069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2bac066f60 0x7f2bac069410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009ca0 tx=0x7f2bb0008040 comp rx=0 tx=0).stop 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 msgr2=0x7f2bc00c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 0x7f2bc00c63a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c840 tx=0x7f2bb8001670 comp rx=0 tx=0).stop 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 shutdown_connections 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2bac066f60 0x7f2bac069410 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc015dbf0 0x7f2bc00c63a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2bc00af440 0x7f2bc00c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 --2- 172.21.15.7:0/2740801426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bc00af810 0x7f2bc00c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 >> 172.21.15.7:0/2740801426 conn(0x7f2bc0069220 msgr2=0x7f2bc00d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:35.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 shutdown_connections 2023-10-07T07:59:35.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.146+0000 7f2bc55d2700 1 -- 172.21.15.7:0/2740801426 wait complete. 2023-10-07T07:59:35.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T07:59:35.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.295+0000 7f1202709700 1 Processor -- start 2023-10-07T07:59:35.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.295+0000 7f1202709700 1 -- start start 2023-10-07T07:59:35.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.296+0000 7f1202709700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11fc067240 0x7f11fc15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.296+0000 7f1202709700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11fc067610 0x7f11fc153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.296+0000 7f1202709700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc06f320 con 0x7f11fc1539c0 2023-10-07T07:59:35.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.296+0000 7f1202709700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc06a500 con 0x7f11fc067610 2023-10-07T07:59:35.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.296+0000 7f1202709700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc153db0 con 0x7f11fc067240 2023-10-07T07:59:35.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.296+0000 7f11fb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11fc067610 0x7f11fc153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.296+0000 7f1200ca6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f11fc1539c0 0x7f11fc0aa980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44088/0 (socket says 172.21.15.7:44088) 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.296+0000 7f1200ca6700 1 -- 172.21.15.7:0/2620366272 learned_addr learned my addr 172.21.15.7:0/2620366272 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.297+0000 7f11fbfff700 1 --2- 172.21.15.7:0/2620366272 >> v2:172.21.15.172:3300/0 conn(0x7f11fc067240 0x7f11fc15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.297+0000 7f11fb7fe700 1 -- 172.21.15.7:0/2620366272 >> v2:172.21.15.172:3300/0 conn(0x7f11fc067240 msgr2=0x7f11fc15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.297+0000 7f11fb7fe700 1 --2- 172.21.15.7:0/2620366272 >> v2:172.21.15.172:3300/0 conn(0x7f11fc067240 0x7f11fc15a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.297+0000 7f11fb7fe700 1 -- 172.21.15.7:0/2620366272 >> v1:172.21.15.7:6789/0 conn(0x7f11fc1539c0 legacy=0x7f11fc0aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.297+0000 7f11fb7fe700 1 -- 172.21.15.7:0/2620366272 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc06e7a0 con 0x7f11fc067610 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.297+0000 7f11fb7fe700 1 --2- 172.21.15.7:0/2620366272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067610 0x7f11fc153480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f11fc067ba0 tx=0x7f11f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe673cf05e0a861c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.297+0000 7f11faffd700 1 -- 172.21.15.7:0/2620366272 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000b6b0 con 0x7f11fc067610 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.297+0000 7f11faffd700 1 -- 172.21.15.7:0/2620366272 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f000b810 con 0x7f11fc067610 2023-10-07T07:59:35.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.297+0000 7f11faffd700 1 -- 172.21.15.7:0/2620366272 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000bbb0 con 0x7f11fc067610 2023-10-07T07:59:35.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.298+0000 7f1202709700 1 -- 172.21.15.7:0/2620366272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067610 msgr2=0x7f11fc153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.298+0000 7f1202709700 1 --2- 172.21.15.7:0/2620366272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067610 0x7f11fc153480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f11fc067ba0 tx=0x7f11f00091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.298+0000 7f1202709700 1 -- 172.21.15.7:0/2620366272 shutdown_connections 2023-10-07T07:59:35.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.298+0000 7f1202709700 1 --2- 172.21.15.7:0/2620366272 >> v2:172.21.15.172:3300/0 conn(0x7f11fc067240 0x7f11fc15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.298+0000 7f1202709700 1 --2- 172.21.15.7:0/2620366272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067610 0x7f11fc153480 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.298+0000 7f1202709700 1 -- 172.21.15.7:0/2620366272 >> 172.21.15.7:0/2620366272 conn(0x7f11fc068100 msgr2=0x7f11fc068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:35.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.298+0000 7f1202709700 1 -- 172.21.15.7:0/2620366272 shutdown_connections 2023-10-07T07:59:35.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.298+0000 7f1202709700 1 -- 172.21.15.7:0/2620366272 wait complete. 2023-10-07T07:59:35.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.299+0000 7f1202709700 1 Processor -- start 2023-10-07T07:59:35.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.299+0000 7f1202709700 1 -- start start 2023-10-07T07:59:35.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f1202709700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067240 0x7f11fc15c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f1202709700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc067610 0x7f11fc06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f1202709700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc1539c0 0x7f11fc140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f1202709700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0c45d0 con 0x7f11fc067240 2023-10-07T07:59:35.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f1202709700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc06cd30 con 0x7f11fc1539c0 2023-10-07T07:59:35.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f1202709700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc140fb0 con 0x7f11fc067610 2023-10-07T07:59:35.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067240 0x7f11fc15c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067240 0x7f11fc15c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37090/0 (socket says 172.21.15.7:37090) 2023-10-07T07:59:35.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f11fbfff700 1 -- 172.21.15.7:0/1610518743 learned_addr learned my addr 172.21.15.7:0/1610518743 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f1200ca6700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc1539c0 0x7f11fc140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.300+0000 7f11fb7fe700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc067610 0x7f11fc06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11fbfff700 1 -- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc067610 msgr2=0x7f11fc06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11fbfff700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc067610 0x7f11fc06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11fbfff700 1 -- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc1539c0 msgr2=0x7f11fc140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11fbfff700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc1539c0 0x7f11fc140a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11fbfff700 1 -- 172.21.15.7:0/1610518743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0009420 con 0x7f11fc067240 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11fb7fe700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc067610 0x7f11fc06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11fbfff700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067240 0x7f11fc15c560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00d8f0 tx=0x7f11ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11e3fff700 1 -- 172.21.15.7:0/1610518743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0125f0 con 0x7f11fc067240 2023-10-07T07:59:35.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11e3fff700 1 -- 172.21.15.7:0/1610518743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec012750 con 0x7f11fc067240 2023-10-07T07:59:35.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc0c6620 con 0x7f11fc067240 2023-10-07T07:59:35.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f11e3fff700 1 -- 172.21.15.7:0/1610518743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec012af0 con 0x7f11fc067240 2023-10-07T07:59:35.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.301+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc0c6be0 con 0x7f11fc067240 2023-10-07T07:59:35.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.302+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc04c3c0 con 0x7f11fc067240 2023-10-07T07:59:35.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.303+0000 7f11e3fff700 1 -- 172.21.15.7:0/1610518743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11ec01e460 con 0x7f11fc067240 2023-10-07T07:59:35.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.303+0000 7f11e3fff700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f11e8066f30 0x7f11e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.303+0000 7f11e3fff700 1 -- 172.21.15.7:0/1610518743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f11ec084aa0 con 0x7f11fc067240 2023-10-07T07:59:35.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.303+0000 7f11fb7fe700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f11e8066f30 0x7f11e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.304+0000 7f11fb7fe700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f11e8066f30 0x7f11e80693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f11f00093f0 tx=0x7f11f0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:35.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.307+0000 7f11e3fff700 1 -- 172.21.15.7:0/1610518743 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11ec051d10 con 0x7f11fc067240 2023-10-07T07:59:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.511+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11fc06ea80 con 0x7f11fc067240 2023-10-07T07:59:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.511+0000 7f11e3fff700 1 -- 172.21.15.7:0/1610518743 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11ec055330 con 0x7f11fc067240 2023-10-07T07:59:35.512 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-07T07:59:35.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f11e8066f30 msgr2=0x7f11e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f11e8066f30 0x7f11e80693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f11f00093f0 tx=0x7f11f0017000 comp rx=0 tx=0).stop 2023-10-07T07:59:35.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067240 msgr2=0x7f11fc15c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067240 0x7f11fc15c560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00d8f0 tx=0x7f11ec008040 comp rx=0 tx=0).stop 2023-10-07T07:59:35.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 shutdown_connections 2023-10-07T07:59:35.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f11e8066f30 0x7f11e80693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc1539c0 0x7f11fc140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11fc067240 0x7f11fc15c560 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 --2- 172.21.15.7:0/1610518743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc067610 0x7f11fc06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 >> 172.21.15.7:0/1610518743 conn(0x7f11fc068100 msgr2=0x7f11fc157650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:35.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 shutdown_connections 2023-10-07T07:59:35.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.513+0000 7f1202709700 1 -- 172.21.15.7:0/1610518743 wait complete. 2023-10-07T07:59:35.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T07:59:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.664+0000 7f729239e700 1 Processor -- start 2023-10-07T07:59:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.664+0000 7f729239e700 1 -- start start 2023-10-07T07:59:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.665+0000 7f729239e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f728c13f8d0 0x7f728c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.665+0000 7f729239e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 0x7f728c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.665+0000 7f729239e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c0b0190 con 0x7f728c15a4f0 2023-10-07T07:59:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.665+0000 7f729239e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c0b0470 con 0x7f728c13f8d0 2023-10-07T07:59:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.665+0000 7f729239e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0b0750 con 0x7f728c0b1440 2023-10-07T07:59:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.665+0000 7f729139c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f728c13f8d0 0x7f728c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.665+0000 7f7290b9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f728c15a4f0 0x7f728c0b0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44100/0 (socket says 172.21.15.7:44100) 2023-10-07T07:59:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.665+0000 7f7290b9b700 1 -- 172.21.15.7:0/1071321313 learned_addr learned my addr 172.21.15.7:0/1071321313 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.665+0000 7f7291b9d700 1 --2- 172.21.15.7:0/1071321313 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 0x7f728c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.666+0000 7f729139c700 1 -- 172.21.15.7:0/1071321313 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 msgr2=0x7f728c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.666+0000 7f729139c700 1 --2- 172.21.15.7:0/1071321313 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 0x7f728c156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.666+0000 7f729139c700 1 -- 172.21.15.7:0/1071321313 >> v1:172.21.15.7:6789/0 conn(0x7f728c15a4f0 legacy=0x7f728c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.666+0000 7f729139c700 1 -- 172.21.15.7:0/1071321313 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c0b0a30 con 0x7f728c13f8d0 2023-10-07T07:59:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.666+0000 7f727ffff700 1 -- 172.21.15.7:0/1071321313 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851669058 0 0) 0x7f728c0b0190 con 0x7f728c15a4f0 2023-10-07T07:59:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.666+0000 7f729139c700 1 --2- 172.21.15.7:0/1071321313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c13f8d0 0x7f728c0afbb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f72880098e0 tx=0x7f72880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2992f20fb65894ec server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.666+0000 7f727ffff700 1 -- 172.21.15.7:0/1071321313 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728800b640 con 0x7f728c13f8d0 2023-10-07T07:59:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.666+0000 7f727ffff700 1 -- 172.21.15.7:0/1071321313 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728800b7a0 con 0x7f728c13f8d0 2023-10-07T07:59:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.666+0000 7f727ffff700 1 -- 172.21.15.7:0/1071321313 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728800bb40 con 0x7f728c13f8d0 2023-10-07T07:59:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.667+0000 7f729239e700 1 -- 172.21.15.7:0/1071321313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c13f8d0 msgr2=0x7f728c0afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.667+0000 7f729239e700 1 --2- 172.21.15.7:0/1071321313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c13f8d0 0x7f728c0afbb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f72880098e0 tx=0x7f72880091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.667+0000 7f729239e700 1 -- 172.21.15.7:0/1071321313 shutdown_connections 2023-10-07T07:59:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.667+0000 7f729239e700 1 --2- 172.21.15.7:0/1071321313 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 0x7f728c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.667+0000 7f729239e700 1 --2- 172.21.15.7:0/1071321313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c13f8d0 0x7f728c0afbb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.667+0000 7f729239e700 1 -- 172.21.15.7:0/1071321313 >> 172.21.15.7:0/1071321313 conn(0x7f728c068100 msgr2=0x7f728c068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.667+0000 7f729239e700 1 -- 172.21.15.7:0/1071321313 shutdown_connections 2023-10-07T07:59:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.667+0000 7f729239e700 1 -- 172.21.15.7:0/1071321313 wait complete. 2023-10-07T07:59:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.668+0000 7f729239e700 1 Processor -- start 2023-10-07T07:59:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.668+0000 7f729239e700 1 -- start start 2023-10-07T07:59:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f729239e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 0x7f728c13e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f729239e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f728c13f8d0 0x7f728c0c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f729239e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c15a4f0 0x7f728c0aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f729239e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c169820 con 0x7f728c15a4f0 2023-10-07T07:59:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f729239e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0c5660 con 0x7f728c13f8d0 2023-10-07T07:59:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f729239e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0c5940 con 0x7f728c0b1440 2023-10-07T07:59:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7291b9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c15a4f0 0x7f728c0aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7291b9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c15a4f0 0x7f728c0aaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37104/0 (socket says 172.21.15.7:37104) 2023-10-07T07:59:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7291b9d700 1 -- 172.21.15.7:0/3614584461 learned_addr learned my addr 172.21.15.7:0/3614584461 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f729139c700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 0x7f728c13e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7290b9b700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f728c13f8d0 0x7f728c0c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7291b9d700 1 -- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 msgr2=0x7f728c13e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7291b9d700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 0x7f728c13e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7291b9d700 1 -- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f728c13f8d0 msgr2=0x7f728c0c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7291b9d700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f728c13f8d0 0x7f728c0c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7291b9d700 1 -- 172.21.15.7:0/3614584461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288009420 con 0x7f728c15a4f0 2023-10-07T07:59:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.669+0000 7f7290b9b700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f728c13f8d0 0x7f728c0c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.670+0000 7f7291b9d700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c15a4f0 0x7f728c0aaf30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7284009d70 tx=0x7f728400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.670+0000 7f727d7fa700 1 -- 172.21.15.7:0/3614584461 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400cb50 con 0x7f728c15a4f0 2023-10-07T07:59:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.670+0000 7f727d7fa700 1 -- 172.21.15.7:0/3614584461 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728400f040 con 0x7f728c15a4f0 2023-10-07T07:59:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.670+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c0c5c20 con 0x7f728c15a4f0 2023-10-07T07:59:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.670+0000 7f727d7fa700 1 -- 172.21.15.7:0/3614584461 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284014480 con 0x7f728c15a4f0 2023-10-07T07:59:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.670+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c0c4510 con 0x7f728c15a4f0 2023-10-07T07:59:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.671+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c1699d0 con 0x7f728c15a4f0 2023-10-07T07:59:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.671+0000 7f727d7fa700 1 -- 172.21.15.7:0/3614584461 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7284014830 con 0x7f728c15a4f0 2023-10-07T07:59:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.672+0000 7f727d7fa700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7270066f30 0x7f72700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.672+0000 7f727d7fa700 1 -- 172.21.15.7:0/3614584461 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7284085150 con 0x7f728c15a4f0 2023-10-07T07:59:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.672+0000 7f729139c700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7270066f30 0x7f72700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.672+0000 7f729139c700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7270066f30 0x7f72700693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f72880093f0 tx=0x7f7288017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.676+0000 7f727d7fa700 1 -- 172.21.15.7:0/3614584461 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72840523c0 con 0x7f728c15a4f0 2023-10-07T07:59:35.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.879+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f728c04c3c0 con 0x7f728c15a4f0 2023-10-07T07:59:35.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.879+0000 7f727d7fa700 1 -- 172.21.15.7:0/3614584461 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f72840559e0 con 0x7f728c15a4f0 2023-10-07T07:59:35.881 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-07T07:59:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7270066f30 msgr2=0x7f72700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7270066f30 0x7f72700693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f72880093f0 tx=0x7f7288017000 comp rx=0 tx=0).stop 2023-10-07T07:59:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c15a4f0 msgr2=0x7f728c0aaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c15a4f0 0x7f728c0aaf30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7284009d70 tx=0x7f728400b040 comp rx=0 tx=0).stop 2023-10-07T07:59:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 shutdown_connections 2023-10-07T07:59:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7270066f30 0x7f72700693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c15a4f0 0x7f728c0aaf30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f728c0b1440 0x7f728c13e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 --2- 172.21.15.7:0/3614584461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f728c13f8d0 0x7f728c0c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 >> 172.21.15.7:0/3614584461 conn(0x7f728c068100 msgr2=0x7f728c0ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 shutdown_connections 2023-10-07T07:59:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:35.882+0000 7f729239e700 1 -- 172.21.15.7:0/3614584461 wait complete. 2023-10-07T07:59:35.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:59:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.027+0000 7f1316b5f700 1 Processor -- start 2023-10-07T07:59:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.028+0000 7f1316b5f700 1 -- start start 2023-10-07T07:59:36.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.028+0000 7f1316b5f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1310157470 0x7f131006b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.028+0000 7f1316b5f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f131006bb60 0x7f13100c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.028+0000 7f1316b5f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13100c8f10 con 0x7f13101570a0 2023-10-07T07:59:36.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.028+0000 7f1316b5f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131006bfe0 con 0x7f131006bb60 2023-10-07T07:59:36.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.028+0000 7f1316b5f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13100c3b80 con 0x7f1310157470 2023-10-07T07:59:36.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f13150fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f131006bb60 0x7f13100c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f13148fb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f13101570a0 0x7f13100c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44110/0 (socket says 172.21.15.7:44110) 2023-10-07T07:59:36.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f13148fb700 1 -- 172.21.15.7:0/1640192515 learned_addr learned my addr 172.21.15.7:0/1640192515 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:36.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f130ffff700 1 --2- 172.21.15.7:0/1640192515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1310157470 0x7f131006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f13150fc700 1 -- 172.21.15.7:0/1640192515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1310157470 msgr2=0x7f131006b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:36.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f13150fc700 1 --2- 172.21.15.7:0/1640192515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1310157470 0x7f131006b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f13150fc700 1 -- 172.21.15.7:0/1640192515 >> v1:172.21.15.7:6789/0 conn(0x7f13101570a0 legacy=0x7f13100c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f13150fc700 1 -- 172.21.15.7:0/1640192515 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100c2f70 con 0x7f131006bb60 2023-10-07T07:59:36.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f13150fc700 1 --2- 172.21.15.7:0/1640192515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 0x7f13100c4010 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1308009e60 tx=0x7f1308009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749bfcc8077329ab server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:36.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f130f7fe700 1 -- 172.21.15.7:0/1640192515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130800e070 con 0x7f131006bb60 2023-10-07T07:59:36.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f130f7fe700 1 -- 172.21.15.7:0/1640192515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130800bc30 con 0x7f131006bb60 2023-10-07T07:59:36.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.029+0000 7f130f7fe700 1 -- 172.21.15.7:0/1640192515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308012430 con 0x7f131006bb60 2023-10-07T07:59:36.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.030+0000 7f1316b5f700 1 -- 172.21.15.7:0/1640192515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 msgr2=0x7f13100c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.030+0000 7f1316b5f700 1 --2- 172.21.15.7:0/1640192515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 0x7f13100c4010 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1308009e60 tx=0x7f1308009780 comp rx=0 tx=0).stop 2023-10-07T07:59:36.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.030+0000 7f1316b5f700 1 -- 172.21.15.7:0/1640192515 shutdown_connections 2023-10-07T07:59:36.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.030+0000 7f1316b5f700 1 --2- 172.21.15.7:0/1640192515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 0x7f13100c4010 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.030+0000 7f1316b5f700 1 --2- 172.21.15.7:0/1640192515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1310157470 0x7f131006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.030+0000 7f1316b5f700 1 -- 172.21.15.7:0/1640192515 >> 172.21.15.7:0/1640192515 conn(0x7f1310067ac0 msgr2=0x7f13100ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:36.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.030+0000 7f1316b5f700 1 -- 172.21.15.7:0/1640192515 shutdown_connections 2023-10-07T07:59:36.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.030+0000 7f1316b5f700 1 -- 172.21.15.7:0/1640192515 wait complete. 2023-10-07T07:59:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.031+0000 7f1316b5f700 1 Processor -- start 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.031+0000 7f1316b5f700 1 -- start start 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.032+0000 7f1316b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 0x7f1310155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.032+0000 7f1316b5f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13101570a0 0x7f13101671d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.032+0000 7f1316b5f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1310157470 0x7f131013ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.032+0000 7f1316b5f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131013fb30 con 0x7f131006bb60 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.032+0000 7f1316b5f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131013ef20 con 0x7f1310157470 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.032+0000 7f1316b5f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131013f200 con 0x7f13101570a0 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.032+0000 7f13148fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 0x7f1310155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.032+0000 7f13148fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 0x7f1310155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37110/0 (socket says 172.21.15.7:37110) 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.032+0000 7f13148fb700 1 -- 172.21.15.7:0/1531146298 learned_addr learned my addr 172.21.15.7:0/1531146298 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f130ffff700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13101570a0 0x7f13101671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f13150fc700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1310157470 0x7f131013ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f13148fb700 1 -- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13101570a0 msgr2=0x7f13101671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f13148fb700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13101570a0 0x7f13101671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f13148fb700 1 -- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1310157470 msgr2=0x7f131013ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f13148fb700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1310157470 0x7f131013ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f13148fb700 1 -- 172.21.15.7:0/1531146298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308009a20 con 0x7f131006bb60 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f13148fb700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 0x7f1310155920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f130400c7c0 tx=0x7f1304007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f130cff9700 1 -- 172.21.15.7:0/1531146298 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304005b30 con 0x7f131006bb60 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f130cff9700 1 -- 172.21.15.7:0/1531146298 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1304005cb0 con 0x7f131006bb60 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131013f4e0 con 0x7f131006bb60 2023-10-07T07:59:36.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f130cff9700 1 -- 172.21.15.7:0/1531146298 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304004a80 con 0x7f131006bb60 2023-10-07T07:59:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.033+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1310156210 con 0x7f131006bb60 2023-10-07T07:59:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.034+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13100716c0 con 0x7f131006bb60 2023-10-07T07:59:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.035+0000 7f130cff9700 1 -- 172.21.15.7:0/1531146298 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f13040070f0 con 0x7f131006bb60 2023-10-07T07:59:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.035+0000 7f130cff9700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f12f4066f30 0x7f12f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.035+0000 7f130cff9700 1 -- 172.21.15.7:0/1531146298 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f13040837d0 con 0x7f131006bb60 2023-10-07T07:59:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.035+0000 7f130ffff700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f12f4066f30 0x7f12f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.036+0000 7f130ffff700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f12f4066f30 0x7f12f40693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f13100c5880 tx=0x7f12fc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.039+0000 7f130cff9700 1 -- 172.21.15.7:0/1531146298 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1304050ac0 con 0x7f131006bb60 2023-10-07T07:59:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.290+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13100c8f10 con 0x7f131006bb60 2023-10-07T07:59:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.291+0000 7f130cff9700 1 -- 172.21.15.7:0/1531146298 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f13100c8f10 con 0x7f131006bb60 2023-10-07T07:59:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:36.292 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/580346028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":580346028},{"type":"v1","addr":"172.21.15.172:6837","nonce":580346028}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3012863722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3012863722},{"type":"v1","addr":"172.21.15.7:6835","nonce":3012863722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4947,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3733075096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3733075096},{"type":"v1","addr":"172.21.15.172:6836","nonce":3733075096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:58:40.003670+0000","modified":"2023-10-07T07:58:43.589885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4959":{"gid":4959,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/786560894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":786560894},{"type":"v1","addr":"172.21.15.7:6834","nonce":786560894}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T07:59:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f12f4066f30 msgr2=0x7f12f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f12f4066f30 0x7f12f40693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f13100c5880 tx=0x7f12fc008040 comp rx=0 tx=0).stop 2023-10-07T07:59:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 msgr2=0x7f1310155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 0x7f1310155920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f130400c7c0 tx=0x7f1304007f70 comp rx=0 tx=0).stop 2023-10-07T07:59:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 shutdown_connections 2023-10-07T07:59:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f12f4066f30 0x7f12f40693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1310157470 0x7f131013ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131006bb60 0x7f1310155920 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 --2- 172.21.15.7:0/1531146298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13101570a0 0x7f13101671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 >> 172.21.15.7:0/1531146298 conn(0x7f1310067ac0 msgr2=0x7f1310139af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 shutdown_connections 2023-10-07T07:59:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.293+0000 7f1316b5f700 1 -- 172.21.15.7:0/1531146298 wait complete. 2023-10-07T07:59:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-10-07T07:59:36.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:59:36.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.439+0000 7f68db0d4700 1 Processor -- start 2023-10-07T07:59:36.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.439+0000 7f68db0d4700 1 -- start start 2023-10-07T07:59:36.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.440+0000 7f68db0d4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68d4159380 0x7f68d40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.440+0000 7f68db0d4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d40b4820 con 0x7f68d40d0250 2023-10-07T07:59:36.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.440+0000 7f68db0d4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d40b5000 con 0x7f68d40b4270 2023-10-07T07:59:36.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.440+0000 7f68db0d4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d40b52e0 con 0x7f68d4159380 2023-10-07T07:59:36.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.440+0000 7f68d8e70700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68d4159380 0x7f68d40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.440+0000 7f68d3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f68d40d0250 0x7f68d40b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44126/0 (socket says 172.21.15.7:44126) 2023-10-07T07:59:36.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.440+0000 7f68d3fff700 1 -- 172.21.15.7:0/616502340 learned_addr learned my addr 172.21.15.7:0/616502340 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:36.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.441+0000 7f68d8e70700 1 -- 172.21.15.7:0/616502340 >> v1:172.21.15.172:6789/0 conn(0x7f68d40b4270 legacy=0x7f68d40ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:36.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.441+0000 7f68d8e70700 1 -- 172.21.15.7:0/616502340 >> v1:172.21.15.7:6789/0 conn(0x7f68d40d0250 legacy=0x7f68d40b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.441+0000 7f68d8e70700 1 -- 172.21.15.7:0/616502340 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d40b55c0 con 0x7f68d4159380 2023-10-07T07:59:36.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.441+0000 7f68d37fe700 1 -- 172.21.15.7:0/616502340 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813743972 0 0) 0x7f68d40b4820 con 0x7f68d40d0250 2023-10-07T07:59:36.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.441+0000 7f68d8e70700 1 --2- 172.21.15.7:0/616502340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 0x7f68d40b5ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f68c80098e0 tx=0x7f68c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc7db4ae0ff3211 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:36.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.441+0000 7f68d37fe700 1 -- 172.21.15.7:0/616502340 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8009e80 con 0x7f68d4159380 2023-10-07T07:59:36.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.441+0000 7f68d37fe700 1 -- 172.21.15.7:0/616502340 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c800b820 con 0x7f68d4159380 2023-10-07T07:59:36.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.441+0000 7f68d37fe700 1 -- 172.21.15.7:0/616502340 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c800bba0 con 0x7f68d4159380 2023-10-07T07:59:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.442+0000 7f68db0d4700 1 -- 172.21.15.7:0/616502340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 msgr2=0x7f68d40b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.442+0000 7f68db0d4700 1 --2- 172.21.15.7:0/616502340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 0x7f68d40b5ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f68c80098e0 tx=0x7f68c80091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.442+0000 7f68db0d4700 1 -- 172.21.15.7:0/616502340 shutdown_connections 2023-10-07T07:59:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.442+0000 7f68db0d4700 1 --2- 172.21.15.7:0/616502340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 0x7f68d40b5ae0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.442+0000 7f68db0d4700 1 -- 172.21.15.7:0/616502340 >> 172.21.15.7:0/616502340 conn(0x7f68d4067be0 msgr2=0x7f68d4067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.442+0000 7f68db0d4700 1 -- 172.21.15.7:0/616502340 shutdown_connections 2023-10-07T07:59:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.442+0000 7f68db0d4700 1 -- 172.21.15.7:0/616502340 wait complete. 2023-10-07T07:59:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.443+0000 7f68db0d4700 1 Processor -- start 2023-10-07T07:59:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.443+0000 7f68db0d4700 1 -- start start 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68db0d4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68d40b4270 0x7f68d414bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68db0d4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68d40d0250 0x7f68d414e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68db0d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 0x7f68d40b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68db0d4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d414f0b0 con 0x7f68d4159380 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68db0d4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d414f390 con 0x7f68d40b4270 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68db0d4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d414f670 con 0x7f68d40d0250 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68d9671700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 0x7f68d40b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68d9671700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 0x7f68d40b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37138/0 (socket says 172.21.15.7:37138) 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68d9671700 1 -- 172.21.15.7:0/1012893542 learned_addr learned my addr 172.21.15.7:0/1012893542 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68d3fff700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68d40d0250 0x7f68d414e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.444+0000 7f68d8e70700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68d40b4270 0x7f68d414bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68d9671700 1 -- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68d40d0250 msgr2=0x7f68d414e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68d9671700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68d40d0250 0x7f68d414e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68d9671700 1 -- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68d40b4270 msgr2=0x7f68d414bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68d9671700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68d40b4270 0x7f68d414bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68d9671700 1 -- 172.21.15.7:0/1012893542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c8009420 con 0x7f68d4159380 2023-10-07T07:59:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68d9671700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 0x7f68d40b1b10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00c930 tx=0x7f68cc0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68d0ff9700 1 -- 172.21.15.7:0/1012893542 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc012630 con 0x7f68d4159380 2023-10-07T07:59:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68d0ff9700 1 -- 172.21.15.7:0/1012893542 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc0127d0 con 0x7f68d4159380 2023-10-07T07:59:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d414f950 con 0x7f68d4159380 2023-10-07T07:59:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68d0ff9700 1 -- 172.21.15.7:0/1012893542 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc012b90 con 0x7f68d4159380 2023-10-07T07:59:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.445+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d414c2b0 con 0x7f68d4159380 2023-10-07T07:59:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.446+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d40716c0 con 0x7f68d4159380 2023-10-07T07:59:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.447+0000 7f68d0ff9700 1 -- 172.21.15.7:0/1012893542 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68cc00a3a0 con 0x7f68d4159380 2023-10-07T07:59:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.447+0000 7f68d0ff9700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f68b8066f30 0x7f68b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.447+0000 7f68d0ff9700 1 -- 172.21.15.7:0/1012893542 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f68cc084210 con 0x7f68d4159380 2023-10-07T07:59:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.447+0000 7f68d8e70700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f68b8066f30 0x7f68b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.448+0000 7f68d8e70700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f68b8066f30 0x7f68b80693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68c80030c0 tx=0x7f68c8016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:36.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.452+0000 7f68d0ff9700 1 -- 172.21.15.7:0/1012893542 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68cc054f10 con 0x7f68d4159380 2023-10-07T07:59:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.659+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68d40b34d0 con 0x7f68d4159380 2023-10-07T07:59:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.660+0000 7f68d0ff9700 1 -- 172.21.15.7:0/1012893542 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f68cc054aa0 con 0x7f68d4159380 2023-10-07T07:59:36.661 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:36.662 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:58:41.578137+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:58:37.928486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T07:58:38.977756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:59:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f68b8066f30 msgr2=0x7f68b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f68b8066f30 0x7f68b80693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68c80030c0 tx=0x7f68c8016000 comp rx=0 tx=0).stop 2023-10-07T07:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 msgr2=0x7f68d40b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 0x7f68d40b1b10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00c930 tx=0x7f68cc0026e0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 shutdown_connections 2023-10-07T07:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f68b8066f30 0x7f68b80693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d4159380 0x7f68d40b1b10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68d40b4270 0x7f68d414bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 --2- 172.21.15.7:0/1012893542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68d40d0250 0x7f68d414e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 >> 172.21.15.7:0/1012893542 conn(0x7f68d4067be0 msgr2=0x7f68d41531a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 shutdown_connections 2023-10-07T07:59:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.662+0000 7f68db0d4700 1 -- 172.21.15.7:0/1012893542 wait complete. 2023-10-07T07:59:36.678 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-07T07:59:36.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:59:36.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.811+0000 7fd05b0e8700 1 Processor -- start 2023-10-07T07:59:36.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.811+0000 7fd05b0e8700 1 -- start start 2023-10-07T07:59:36.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.811+0000 7fd05b0e8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd054140a80 0x7fd05406ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.811+0000 7fd05b0e8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0541582c0 0x7fd054066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd05b0e8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05406e2c0 con 0x7fd054067530 2023-10-07T07:59:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd05b0e8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05406f250 con 0x7fd054140a80 2023-10-07T07:59:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd05b0e8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd054067950 con 0x7fd0541582c0 2023-10-07T07:59:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd058e84700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd054140a80 0x7fd05406ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd058e84700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd054140a80 0x7fd05406ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37152/0 (socket says 172.21.15.7:37152) 2023-10-07T07:59:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd058e84700 1 -- 172.21.15.7:0/23288418 learned_addr learned my addr 172.21.15.7:0/23288418 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd053fff700 1 --2- 172.21.15.7:0/23288418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0541582c0 0x7fd054066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd058e84700 1 -- 172.21.15.7:0/23288418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0541582c0 msgr2=0x7fd054066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd058e84700 1 --2- 172.21.15.7:0/23288418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0541582c0 0x7fd054066ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd058e84700 1 -- 172.21.15.7:0/23288418 >> v1:172.21.15.172:6789/0 conn(0x7fd054067530 legacy=0x7fd05406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd058e84700 1 -- 172.21.15.7:0/23288418 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540c7940 con 0x7fd054140a80 2023-10-07T07:59:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd053fff700 1 --2- 172.21.15.7:0/23288418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0541582c0 0x7fd054066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.812+0000 7fd058e84700 1 --2- 172.21.15.7:0/23288418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054140a80 0x7fd05406ed10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd0480099b0 tx=0x7fd048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208e78aa50ee83d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd0537fe700 1 -- 172.21.15.7:0/23288418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04800b860 con 0x7fd054140a80 2023-10-07T07:59:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd0537fe700 1 -- 172.21.15.7:0/23288418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04800b9c0 con 0x7fd054140a80 2023-10-07T07:59:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd0537fe700 1 -- 172.21.15.7:0/23288418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04800bd60 con 0x7fd054140a80 2023-10-07T07:59:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd05b0e8700 1 -- 172.21.15.7:0/23288418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054140a80 msgr2=0x7fd05406ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd05b0e8700 1 --2- 172.21.15.7:0/23288418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054140a80 0x7fd05406ed10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd0480099b0 tx=0x7fd048009250 comp rx=0 tx=0).stop 2023-10-07T07:59:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd05b0e8700 1 -- 172.21.15.7:0/23288418 shutdown_connections 2023-10-07T07:59:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd05b0e8700 1 --2- 172.21.15.7:0/23288418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054140a80 0x7fd05406ed10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd05b0e8700 1 --2- 172.21.15.7:0/23288418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0541582c0 0x7fd054066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd05b0e8700 1 -- 172.21.15.7:0/23288418 >> 172.21.15.7:0/23288418 conn(0x7fd054068bd0 msgr2=0x7fd0540ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd05b0e8700 1 -- 172.21.15.7:0/23288418 shutdown_connections 2023-10-07T07:59:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.813+0000 7fd05b0e8700 1 -- 172.21.15.7:0/23288418 wait complete. 2023-10-07T07:59:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.814+0000 7fd05b0e8700 1 Processor -- start 2023-10-07T07:59:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.815+0000 7fd05b0e8700 1 -- start start 2023-10-07T07:59:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.815+0000 7fd05b0e8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054067530 0x7fd0540b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd05b0e8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd054140a80 0x7fd0541569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd05b0e8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0541582c0 0x7fd05413de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd05b0e8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540c6f60 con 0x7fd054067530 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd05b0e8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540b2090 con 0x7fd0541582c0 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd05b0e8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540c6070 con 0x7fd054140a80 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd058e84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054067530 0x7fd0540b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd058e84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054067530 0x7fd0540b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37156/0 (socket says 172.21.15.7:37156) 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd058e84700 1 -- 172.21.15.7:0/1687610153 learned_addr learned my addr 172.21.15.7:0/1687610153 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd053fff700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd054140a80 0x7fd0541569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd059685700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0541582c0 0x7fd05413de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd058e84700 1 -- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd054140a80 msgr2=0x7fd0541569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd058e84700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd054140a80 0x7fd0541569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd058e84700 1 -- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0541582c0 msgr2=0x7fd05413de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd058e84700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0541582c0 0x7fd05413de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd058e84700 1 -- 172.21.15.7:0/1687610153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0480094f0 con 0x7fd054067530 2023-10-07T07:59:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd058e84700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054067530 0x7fd0540b1b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd048009290 tx=0x7fd048017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd050ff9700 1 -- 172.21.15.7:0/1687610153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048026480 con 0x7fd054067530 2023-10-07T07:59:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd050ff9700 1 -- 172.21.15.7:0/1687610153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04801f070 con 0x7fd054067530 2023-10-07T07:59:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0540c6350 con 0x7fd054067530 2023-10-07T07:59:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd050ff9700 1 -- 172.21.15.7:0/1687610153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048026830 con 0x7fd054067530 2023-10-07T07:59:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.816+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0540c6910 con 0x7fd054067530 2023-10-07T07:59:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.817+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05406ba20 con 0x7fd054067530 2023-10-07T07:59:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.818+0000 7fd050ff9700 1 -- 172.21.15.7:0/1687610153 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd04801a430 con 0x7fd054067530 2023-10-07T07:59:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.818+0000 7fd050ff9700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd038066f30 0x7fd0380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.818+0000 7fd050ff9700 1 -- 172.21.15.7:0/1687610153 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd048021070 con 0x7fd054067530 2023-10-07T07:59:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.818+0000 7fd053fff700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd038066f30 0x7fd0380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.819+0000 7fd053fff700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd038066f30 0x7fd0380693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd040006f00 tx=0x7fd040008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:36.823+0000 7fd050ff9700 1 -- 172.21.15.7:0/1687610153 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd048053720 con 0x7fd054067530 2023-10-07T07:59:37.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.070+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0540716c0 con 0x7fd054067530 2023-10-07T07:59:37.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.071+0000 7fd050ff9700 1 -- 172.21.15.7:0/1687610153 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fd048056d40 con 0x7fd054067530 2023-10-07T07:59:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:37.072 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/580346028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":580346028},{"type":"v1","addr":"172.21.15.172:6837","nonce":580346028}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3012863722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3012863722},{"type":"v1","addr":"172.21.15.7:6835","nonce":3012863722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4947,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3733075096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3733075096},{"type":"v1","addr":"172.21.15.172:6836","nonce":3733075096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:58:40.003670+0000","modified":"2023-10-07T07:58:43.589885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4959":{"gid":4959,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/786560894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":786560894},{"type":"v1","addr":"172.21.15.7:6834","nonce":786560894}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T07:59:37.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd038066f30 msgr2=0x7fd0380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd038066f30 0x7fd0380693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd040006f00 tx=0x7fd040008040 comp rx=0 tx=0).stop 2023-10-07T07:59:37.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054067530 msgr2=0x7fd0540b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054067530 0x7fd0540b1b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd048009290 tx=0x7fd048017000 comp rx=0 tx=0).stop 2023-10-07T07:59:37.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 shutdown_connections 2023-10-07T07:59:37.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd038066f30 0x7fd0380693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0541582c0 0x7fd05413de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd054067530 0x7fd0540b1b50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 --2- 172.21.15.7:0/1687610153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd054140a80 0x7fd0541569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 >> 172.21.15.7:0/1687610153 conn(0x7fd054068bd0 msgr2=0x7fd05413f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:37.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 shutdown_connections 2023-10-07T07:59:37.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.073+0000 7fd05b0e8700 1 -- 172.21.15.7:0/1687610153 wait complete. 2023-10-07T07:59:37.076 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-10-07T07:59:37.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:59:37.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.218+0000 7f52c3f8d700 1 Processor -- start 2023-10-07T07:59:37.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.219+0000 7f52c3f8d700 1 -- start start 2023-10-07T07:59:37.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c3f8d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 0x7f52bc0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c3f8d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52bc0b4270 0x7f52bc0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c3f8d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52bc0d0be0 con 0x7f52bc0d0250 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c3f8d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52bc0b4820 con 0x7f52bc0b4270 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c3f8d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc0b5000 con 0x7f52bc159380 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c252a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52bc0b4270 0x7f52bc0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c1528700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f52bc0d0250 0x7f52bc0b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44130/0 (socket says 172.21.15.7:44130) 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c1528700 1 -- 172.21.15.7:0/3635671026 learned_addr learned my addr 172.21.15.7:0/3635671026 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c1d29700 1 --2- 172.21.15.7:0/3635671026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 0x7f52bc0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c252a700 1 -- 172.21.15.7:0/3635671026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 msgr2=0x7f52bc0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c252a700 1 --2- 172.21.15.7:0/3635671026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 0x7f52bc0b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c252a700 1 -- 172.21.15.7:0/3635671026 >> v1:172.21.15.7:6789/0 conn(0x7f52bc0d0250 legacy=0x7f52bc0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c252a700 1 -- 172.21.15.7:0/3635671026 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc0b52e0 con 0x7f52bc0b4270 2023-10-07T07:59:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c0d27700 1 -- 172.21.15.7:0/3635671026 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526673618 0 0) 0x7f52bc0d0be0 con 0x7f52bc0d0250 2023-10-07T07:59:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.220+0000 7f52c252a700 1 --2- 172.21.15.7:0/3635671026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0b4270 0x7f52bc0d06a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e60 tx=0x7f52b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afd93abcfde7387 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c0d27700 1 -- 172.21.15.7:0/3635671026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400e070 con 0x7f52bc0b4270 2023-10-07T07:59:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c0d27700 1 -- 172.21.15.7:0/3635671026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b400bb60 con 0x7f52bc0b4270 2023-10-07T07:59:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c0d27700 1 -- 172.21.15.7:0/3635671026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4012430 con 0x7f52bc0b4270 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3635671026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0b4270 msgr2=0x7f52bc0d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c3f8d700 1 --2- 172.21.15.7:0/3635671026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0b4270 0x7f52bc0d06a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e60 tx=0x7f52b4009780 comp rx=0 tx=0).stop 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3635671026 shutdown_connections 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c3f8d700 1 --2- 172.21.15.7:0/3635671026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0b4270 0x7f52bc0d06a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c3f8d700 1 --2- 172.21.15.7:0/3635671026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 0x7f52bc0b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3635671026 >> 172.21.15.7:0/3635671026 conn(0x7f52bc067be0 msgr2=0x7f52bc067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3635671026 shutdown_connections 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.221+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3635671026 wait complete. 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.222+0000 7f52c3f8d700 1 Processor -- start 2023-10-07T07:59:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.223+0000 7f52c3f8d700 1 -- start start 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.223+0000 7f52c3f8d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52bc0b4270 0x7f52bc16ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.223+0000 7f52c3f8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0d0250 0x7f52bc1461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.223+0000 7f52c3f8d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 0x7f52bc15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.223+0000 7f52c3f8d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc16b660 con 0x7f52bc0d0250 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.223+0000 7f52c3f8d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc16b940 con 0x7f52bc0b4270 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.223+0000 7f52c3f8d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc16bc20 con 0x7f52bc159380 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.223+0000 7f52c1528700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0d0250 0x7f52bc1461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c1528700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0d0250 0x7f52bc1461b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37176/0 (socket says 172.21.15.7:37176) 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c1528700 1 -- 172.21.15.7:0/3713130350 learned_addr learned my addr 172.21.15.7:0/3713130350 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c1d29700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52bc0b4270 0x7f52bc16ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c252a700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 0x7f52bc15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c1528700 1 -- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 msgr2=0x7f52bc15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c1528700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 0x7f52bc15b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c1528700 1 -- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52bc0b4270 msgr2=0x7f52bc16ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c1528700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52bc0b4270 0x7f52bc16ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c1528700 1 -- 172.21.15.7:0/3713130350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4009a20 con 0x7f52bc0d0250 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c1528700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0d0250 0x7f52bc1461b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00c7e0 tx=0x7f52ac007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52b1ffb700 1 -- 172.21.15.7:0/3713130350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac005b30 con 0x7f52bc0d0250 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52b1ffb700 1 -- 172.21.15.7:0/3713130350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac005c90 con 0x7f52bc0d0250 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc16bf00 con 0x7f52bc0d0250 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.224+0000 7f52b1ffb700 1 -- 172.21.15.7:0/3713130350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac007300 con 0x7f52bc0d0250 2023-10-07T07:59:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.225+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc16b2e0 con 0x7f52bc0d0250 2023-10-07T07:59:37.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.226+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52bc16c420 con 0x7f52bc0d0250 2023-10-07T07:59:37.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.226+0000 7f52b1ffb700 1 -- 172.21.15.7:0/3713130350 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52ac01b400 con 0x7f52bc0d0250 2023-10-07T07:59:37.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.226+0000 7f52b1ffb700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52a0066f30 0x7f52a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.226+0000 7f52b1ffb700 1 -- 172.21.15.7:0/3713130350 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f52ac084040 con 0x7f52bc0d0250 2023-10-07T07:59:37.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.226+0000 7f52c1d29700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52a0066f30 0x7f52a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.227+0000 7f52c1d29700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52a0066f30 0x7f52a00693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009ca0 tx=0x7f52b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:37.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.231+0000 7f52b1ffb700 1 -- 172.21.15.7:0/3713130350 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52bc16c420 con 0x7f52bc0d0250 2023-10-07T07:59:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.477+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52bc16c420 con 0x7f52bc0d0250 2023-10-07T07:59:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.478+0000 7f52b1ffb700 1 -- 172.21.15.7:0/3713130350 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f52bc16c420 con 0x7f52bc0d0250 2023-10-07T07:59:37.479 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:37.479 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/580346028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":580346028},{"type":"v1","addr":"172.21.15.172:6837","nonce":580346028}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3012863722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3012863722},{"type":"v1","addr":"172.21.15.7:6835","nonce":3012863722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4947,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3733075096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3733075096},{"type":"v1","addr":"172.21.15.172:6836","nonce":3733075096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:58:40.003670+0000","modified":"2023-10-07T07:58:43.589885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4959":{"gid":4959,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/786560894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":786560894},{"type":"v1","addr":"172.21.15.7:6834","nonce":786560894}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T07:59:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52a0066f30 msgr2=0x7f52a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52a0066f30 0x7f52a00693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009ca0 tx=0x7f52b8008040 comp rx=0 tx=0).stop 2023-10-07T07:59:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0d0250 msgr2=0x7f52bc1461b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0d0250 0x7f52bc1461b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00c7e0 tx=0x7f52ac007df0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 shutdown_connections 2023-10-07T07:59:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52a0066f30 0x7f52a00693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52bc159380 0x7f52bc15b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52bc0b4270 0x7f52bc16ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 --2- 172.21.15.7:0/3713130350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52bc0d0250 0x7f52bc1461b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 >> 172.21.15.7:0/3713130350 conn(0x7f52bc067be0 msgr2=0x7f52bc0ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 shutdown_connections 2023-10-07T07:59:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.480+0000 7f52c3f8d700 1 -- 172.21.15.7:0/3713130350 wait complete. 2023-10-07T07:59:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-10-07T07:59:37.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:59:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.629+0000 7f1c9ffd1700 1 Processor -- start 2023-10-07T07:59:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.629+0000 7f1c9ffd1700 1 -- start start 2023-10-07T07:59:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.630+0000 7f1c9ffd1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c980ced30 0x7f1c980c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.630+0000 7f1c9ffd1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c981596e0 0x7f1c9806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.630+0000 7f1c9ffd1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c98068100 con 0x7f1c98159310 2023-10-07T07:59:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.630+0000 7f1c9ffd1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c98068350 con 0x7f1c981596e0 2023-10-07T07:59:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.630+0000 7f1c9ffd1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98068630 con 0x7f1c980ced30 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.630+0000 7f1c9e56e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c981596e0 0x7f1c9806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.630+0000 7f1c9dd6d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1c98159310 0x7f1c980c3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44132/0 (socket says 172.21.15.7:44132) 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.630+0000 7f1c9dd6d700 1 -- 172.21.15.7:0/2852286715 learned_addr learned my addr 172.21.15.7:0/2852286715 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9d56c700 1 --2- 172.21.15.7:0/2852286715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c980ced30 0x7f1c980c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9e56e700 1 -- 172.21.15.7:0/2852286715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c980ced30 msgr2=0x7f1c980c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9e56e700 1 --2- 172.21.15.7:0/2852286715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c980ced30 0x7f1c980c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9e56e700 1 -- 172.21.15.7:0/2852286715 >> v1:172.21.15.7:6789/0 conn(0x7f1c98159310 legacy=0x7f1c980c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9e56e700 1 -- 172.21.15.7:0/2852286715 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c98068910 con 0x7f1c981596e0 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9cd6b700 1 -- 172.21.15.7:0/2852286715 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335100144 0 0) 0x7f1c98068100 con 0x7f1c98159310 2023-10-07T07:59:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9e56e700 1 --2- 172.21.15.7:0/2852286715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c981596e0 0x7f1c9806b620 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009e60 tx=0x7f1c90009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50339d771880c243 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9cd6b700 1 -- 172.21.15.7:0/2852286715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000e070 con 0x7f1c981596e0 2023-10-07T07:59:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9cd6b700 1 -- 172.21.15.7:0/2852286715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9000bc30 con 0x7f1c981596e0 2023-10-07T07:59:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.631+0000 7f1c9cd6b700 1 -- 172.21.15.7:0/2852286715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90012430 con 0x7f1c981596e0 2023-10-07T07:59:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.632+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2852286715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c981596e0 msgr2=0x7f1c9806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.632+0000 7f1c9ffd1700 1 --2- 172.21.15.7:0/2852286715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c981596e0 0x7f1c9806b620 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009e60 tx=0x7f1c90009780 comp rx=0 tx=0).stop 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.632+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2852286715 shutdown_connections 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.632+0000 7f1c9ffd1700 1 --2- 172.21.15.7:0/2852286715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c981596e0 0x7f1c9806b620 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.632+0000 7f1c9ffd1700 1 --2- 172.21.15.7:0/2852286715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c980ced30 0x7f1c980c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.632+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2852286715 >> 172.21.15.7:0/2852286715 conn(0x7f1c98067ac0 msgr2=0x7f1c980ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.632+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2852286715 shutdown_connections 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.632+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2852286715 wait complete. 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.633+0000 7f1c9ffd1700 1 Processor -- start 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.633+0000 7f1c9ffd1700 1 -- start start 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9ffd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c98159310 0x7f1c9815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9ffd1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c981596e0 0x7f1c9816d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9ffd1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c9815d470 0x7f1c981534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9ffd1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980c5150 con 0x7f1c98159310 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9ffd1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980c5cd0 con 0x7f1c9815d470 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9ffd1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9815b870 con 0x7f1c981596e0 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9dd6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c98159310 0x7f1c9815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9dd6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c98159310 0x7f1c9815c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37202/0 (socket says 172.21.15.7:37202) 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9dd6d700 1 -- 172.21.15.7:0/2128050343 learned_addr learned my addr 172.21.15.7:0/2128050343 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9e56e700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c9815d470 0x7f1c981534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9d56c700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c981596e0 0x7f1c9816d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9dd6d700 1 -- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c981596e0 msgr2=0x7f1c9816d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9dd6d700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c981596e0 0x7f1c9816d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9dd6d700 1 -- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c9815d470 msgr2=0x7f1c981534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9dd6d700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c9815d470 0x7f1c981534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9dd6d700 1 -- 172.21.15.7:0/2128050343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c90009a20 con 0x7f1c98159310 2023-10-07T07:59:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.634+0000 7f1c9dd6d700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c98159310 0x7f1c9815c590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c810 tx=0x7f1c94007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.635+0000 7f1c8dffb700 1 -- 172.21.15.7:0/2128050343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94005b30 con 0x7f1c98159310 2023-10-07T07:59:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.635+0000 7f1c8dffb700 1 -- 172.21.15.7:0/2128050343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94005c90 con 0x7f1c98159310 2023-10-07T07:59:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.635+0000 7f1c8dffb700 1 -- 172.21.15.7:0/2128050343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94007300 con 0x7f1c98159310 2023-10-07T07:59:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.635+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c980c53a0 con 0x7f1c98159310 2023-10-07T07:59:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.635+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c980c5960 con 0x7f1c98159310 2023-10-07T07:59:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.636+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c980716c0 con 0x7f1c98159310 2023-10-07T07:59:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.636+0000 7f1c8dffb700 1 -- 172.21.15.7:0/2128050343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c9401b400 con 0x7f1c98159310 2023-10-07T07:59:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.636+0000 7f1c8dffb700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1c7c066f60 0x7f1c7c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.637+0000 7f1c8dffb700 1 -- 172.21.15.7:0/2128050343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c94084000 con 0x7f1c98159310 2023-10-07T07:59:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.637+0000 7f1c9d56c700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1c7c066f60 0x7f1c7c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:37.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.637+0000 7f1c9d56c700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1c7c066f60 0x7f1c7c069410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c9816c870 tx=0x7f1c88008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.641+0000 7f1c8dffb700 1 -- 172.21.15.7:0/2128050343 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c940512f0 con 0x7f1c98159310 2023-10-07T07:59:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.845+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c980cea10 con 0x7f1c98159310 2023-10-07T07:59:37.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.846+0000 7f1c8dffb700 1 -- 172.21.15.7:0/2128050343 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f1c94054910 con 0x7f1c98159310 2023-10-07T07:59:37.847 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:37.847 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:58:41.578137+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:58:37.928486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T07:58:38.977756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:59:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.848+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1c7c066f60 msgr2=0x7f1c7c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.848+0000 7f1c9ffd1700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1c7c066f60 0x7f1c7c069410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c9816c870 tx=0x7f1c88008040 comp rx=0 tx=0).stop 2023-10-07T07:59:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c98159310 msgr2=0x7f1c9815c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c98159310 0x7f1c9815c590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c810 tx=0x7f1c94007df0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 shutdown_connections 2023-10-07T07:59:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1c7c066f60 0x7f1c7c069410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c9815d470 0x7f1c981534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c98159310 0x7f1c9815c590 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 --2- 172.21.15.7:0/2128050343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c981596e0 0x7f1c9816d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 >> 172.21.15.7:0/2128050343 conn(0x7f1c98067ac0 msgr2=0x7f1c980cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 shutdown_connections 2023-10-07T07:59:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.849+0000 7f1c9ffd1700 1 -- 172.21.15.7:0/2128050343 wait complete. 2023-10-07T07:59:37.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-07T07:59:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.996+0000 7fae930e2700 1 Processor -- start 2023-10-07T07:59:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.996+0000 7fae930e2700 1 -- start start 2023-10-07T07:59:37.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae930e2700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae8c1592c0 0x7fae8c06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae930e2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 0x7fae8c06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:37.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae930e2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8c06bb70 con 0x7fae8c159690 2023-10-07T07:59:37.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae930e2700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8c0cda10 con 0x7fae8c1592c0 2023-10-07T07:59:37.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae930e2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c06af60 con 0x7fae8c0cb290 2023-10-07T07:59:37.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae8bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fae8c159690 0x7fae8c0cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44136/0 (socket says 172.21.15.7:44136) 2023-10-07T07:59:37.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae8bfff700 1 -- 172.21.15.7:0/447735051 learned_addr learned my addr 172.21.15.7:0/447735051 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:37.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae90e7e700 1 --2- 172.21.15.7:0/447735051 >> v2:172.21.15.172:3300/0 conn(0x7fae8c1592c0 0x7fae8c06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae9167f700 1 --2- 172.21.15.7:0/447735051 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 0x7fae8c06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae8b7fe700 1 -- 172.21.15.7:0/447735051 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345142067 0 0) 0x7fae8c06bb70 con 0x7fae8c159690 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae8b7fe700 1 -- 172.21.15.7:0/447735051 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7c003570 con 0x7fae8c159690 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae8b7fe700 1 -- 172.21.15.7:0/447735051 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3648761709 0 0) 0x7fae7c003570 con 0x7fae8c159690 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae9167f700 1 -- 172.21.15.7:0/447735051 >> v2:172.21.15.172:3300/0 conn(0x7fae8c1592c0 msgr2=0x7fae8c06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae9167f700 1 --2- 172.21.15.7:0/447735051 >> v2:172.21.15.172:3300/0 conn(0x7fae8c1592c0 0x7fae8c06a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae9167f700 1 -- 172.21.15.7:0/447735051 >> v1:172.21.15.7:6789/0 conn(0x7fae8c159690 legacy=0x7fae8c0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae9167f700 1 -- 172.21.15.7:0/447735051 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c06b240 con 0x7fae8c0cb290 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.997+0000 7fae90e7e700 1 --2- 172.21.15.7:0/447735051 >> v2:172.21.15.172:3300/0 conn(0x7fae8c1592c0 0x7fae8c06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae9167f700 1 --2- 172.21.15.7:0/447735051 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 0x7fae8c06a930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fae84009e60 tx=0x7fae84009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c50df00997df99dd server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae8b7fe700 1 -- 172.21.15.7:0/447735051 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400e070 con 0x7fae8c0cb290 2023-10-07T07:59:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae8b7fe700 1 -- 172.21.15.7:0/447735051 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8400b7a0 con 0x7fae8c0cb290 2023-10-07T07:59:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae8b7fe700 1 -- 172.21.15.7:0/447735051 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78001470 con 0x7fae8c0cb290 2023-10-07T07:59:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae930e2700 1 -- 172.21.15.7:0/447735051 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 msgr2=0x7fae8c06a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae930e2700 1 --2- 172.21.15.7:0/447735051 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 0x7fae8c06a930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fae84009e60 tx=0x7fae84009780 comp rx=0 tx=0).stop 2023-10-07T07:59:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae930e2700 1 -- 172.21.15.7:0/447735051 shutdown_connections 2023-10-07T07:59:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae930e2700 1 --2- 172.21.15.7:0/447735051 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 0x7fae8c06a930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae930e2700 1 --2- 172.21.15.7:0/447735051 >> v2:172.21.15.172:3300/0 conn(0x7fae8c1592c0 0x7fae8c06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae930e2700 1 -- 172.21.15.7:0/447735051 >> 172.21.15.7:0/447735051 conn(0x7fae8c068bd0 msgr2=0x7fae8c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae930e2700 1 -- 172.21.15.7:0/447735051 shutdown_connections 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:37.998+0000 7fae930e2700 1 -- 172.21.15.7:0/447735051 wait complete. 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.000+0000 7fae930e2700 1 Processor -- start 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.000+0000 7fae930e2700 1 -- start start 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae930e2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 0x7fae8c144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae930e2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae8c1592c0 0x7fae8c0b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae930e2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae8c159690 0x7fae8c06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae930e2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c0ac430 con 0x7fae8c159690 2023-10-07T07:59:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae930e2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c0ab860 con 0x7fae8c1592c0 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae930e2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c0abb00 con 0x7fae8c0cb290 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae9167f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae8c159690 0x7fae8c06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae9167f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae8c159690 0x7fae8c06a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37204/0 (socket says 172.21.15.7:37204) 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae9167f700 1 -- 172.21.15.7:0/618244669 learned_addr learned my addr 172.21.15.7:0/618244669 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae90e7e700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 0x7fae8c144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae8bfff700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae8c1592c0 0x7fae8c0b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae9167f700 1 -- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 msgr2=0x7fae8c144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae9167f700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 0x7fae8c144b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae9167f700 1 -- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae8c1592c0 msgr2=0x7fae8c0b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae9167f700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae8c1592c0 0x7fae8c0b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae9167f700 1 -- 172.21.15.7:0/618244669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80009420 con 0x7fae8c159690 2023-10-07T07:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.001+0000 7fae9167f700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae8c159690 0x7fae8c06a2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0afaf0 tx=0x7fae84018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.002+0000 7fae88ff9700 1 -- 172.21.15.7:0/618244669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84012bc0 con 0x7fae8c159690 2023-10-07T07:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.002+0000 7fae88ff9700 1 -- 172.21.15.7:0/618244669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae84012d60 con 0x7fae8c159690 2023-10-07T07:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.002+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae84009a20 con 0x7fae8c159690 2023-10-07T07:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.002+0000 7fae88ff9700 1 -- 172.21.15.7:0/618244669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8401e680 con 0x7fae8c159690 2023-10-07T07:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.002+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c0ac0c0 con 0x7fae8c159690 2023-10-07T07:59:38.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.003+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c0ac5a0 con 0x7fae8c159690 2023-10-07T07:59:38.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.003+0000 7fae88ff9700 1 -- 172.21.15.7:0/618244669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae8401e9f0 con 0x7fae8c159690 2023-10-07T07:59:38.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.003+0000 7fae88ff9700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fae70066f30 0x7fae700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.004+0000 7fae88ff9700 1 -- 172.21.15.7:0/618244669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae84086510 con 0x7fae8c159690 2023-10-07T07:59:38.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.004+0000 7fae90e7e700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fae70066f30 0x7fae700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.004+0000 7fae90e7e700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fae70066f30 0x7fae700693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae80009ea0 tx=0x7fae800091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.008+0000 7fae88ff9700 1 -- 172.21.15.7:0/618244669 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae84053780 con 0x7fae8c159690 2023-10-07T07:59:38.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.258+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fae8c0716c0 con 0x7fae8c159690 2023-10-07T07:59:38.260 INFO:tasks.ceph.mon.a.smithi007.stderr:2023-10-07T07:59:38.259+0000 7feeb1c30700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T07:59:38.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.263+0000 7fae88ff9700 1 -- 172.21.15.7:0/618244669 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fae84056da0 con 0x7fae8c159690 2023-10-07T07:59:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fae70066f30 msgr2=0x7fae700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fae70066f30 0x7fae700693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae80009ea0 tx=0x7fae800091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae8c159690 msgr2=0x7fae8c06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae8c159690 0x7fae8c06a2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0afaf0 tx=0x7fae84018000 comp rx=0 tx=0).stop 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 shutdown_connections 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fae70066f30 0x7fae700693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae8c159690 0x7fae8c06a2f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae8c0cb290 0x7fae8c144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 --2- 172.21.15.7:0/618244669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae8c1592c0 0x7fae8c0b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 >> 172.21.15.7:0/618244669 conn(0x7fae8c068bd0 msgr2=0x7fae8c0c1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 shutdown_connections 2023-10-07T07:59:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.265+0000 7fae930e2700 1 -- 172.21.15.7:0/618244669 wait complete. 2023-10-07T07:59:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-07T07:59:38.283 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-07T07:59:38.304 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2023-10-07T07:59:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.419+0000 7fb2b8399700 1 Processor -- start 2023-10-07T07:59:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.419+0000 7fb2b8399700 1 -- start start 2023-10-07T07:59:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.420+0000 7fb2b8399700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2b0157300 0x7fb2b006e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.420+0000 7fb2b8399700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0067610 0x7fb2b006eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.420+0000 7fb2b8399700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b00c8020 con 0x7fb2b0067240 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.420+0000 7fb2b8399700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b006f3b0 con 0x7fb2b0157300 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.420+0000 7fb2b8399700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b006f0e0 con 0x7fb2b0067610 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5934700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2b0157300 0x7fb2b006e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b6135700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb2b0067240 0x7fb2b0156bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44144/0 (socket says 172.21.15.7:44144) 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b6135700 1 -- 172.21.15.7:0/872467569 learned_addr learned my addr 172.21.15.7:0/872467569 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5934700 1 -- 172.21.15.7:0/872467569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0067610 msgr2=0x7fb2b006eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b6936700 1 --2- 172.21.15.7:0/872467569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0067610 0x7fb2b006eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5934700 1 --2- 172.21.15.7:0/872467569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0067610 0x7fb2b006eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5133700 1 -- 172.21.15.7:0/872467569 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394273763 0 0) 0x7fb2b00c8020 con 0x7fb2b0067240 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5934700 1 -- 172.21.15.7:0/872467569 >> v1:172.21.15.7:6789/0 conn(0x7fb2b0067240 legacy=0x7fb2b0156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5934700 1 -- 172.21.15.7:0/872467569 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b00c76f0 con 0x7fb2b0157300 2023-10-07T07:59:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b6936700 1 --2- 172.21.15.7:0/872467569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0067610 0x7fb2b006eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:38.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5934700 1 --2- 172.21.15.7:0/872467569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0157300 0x7fb2b006e660 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0098e0 tx=0x7fb2ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18090a3a12d8fd12 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:38.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5133700 1 -- 172.21.15.7:0/872467569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac009e80 con 0x7fb2b0157300 2023-10-07T07:59:38.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5133700 1 -- 172.21.15.7:0/872467569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac00b860 con 0x7fb2b0157300 2023-10-07T07:59:38.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.421+0000 7fb2b5133700 1 -- 172.21.15.7:0/872467569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00bbe0 con 0x7fb2b0157300 2023-10-07T07:59:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.422+0000 7fb2b8399700 1 -- 172.21.15.7:0/872467569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0157300 msgr2=0x7fb2b006e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.422+0000 7fb2b8399700 1 --2- 172.21.15.7:0/872467569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0157300 0x7fb2b006e660 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0098e0 tx=0x7fb2ac0091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.422+0000 7fb2b8399700 1 -- 172.21.15.7:0/872467569 shutdown_connections 2023-10-07T07:59:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.422+0000 7fb2b8399700 1 --2- 172.21.15.7:0/872467569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0067610 0x7fb2b006eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.422+0000 7fb2b8399700 1 --2- 172.21.15.7:0/872467569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0157300 0x7fb2b006e660 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.422+0000 7fb2b8399700 1 -- 172.21.15.7:0/872467569 >> 172.21.15.7:0/872467569 conn(0x7fb2b0068100 msgr2=0x7fb2b0068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.422+0000 7fb2b8399700 1 -- 172.21.15.7:0/872467569 shutdown_connections 2023-10-07T07:59:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.422+0000 7fb2b8399700 1 -- 172.21.15.7:0/872467569 wait complete. 2023-10-07T07:59:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.423+0000 7fb2b8399700 1 Processor -- start 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.423+0000 7fb2b8399700 1 -- start start 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b8399700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b0067240 0x7fb2b0164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b8399700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0067610 0x7fb2b00b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b8399700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0157300 0x7fb2b00b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b8399700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b00ae680 con 0x7fb2b0067610 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b8399700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01677b0 con 0x7fb2b0067240 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b8399700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b0167a90 con 0x7fb2b0157300 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b5934700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0067610 0x7fb2b00b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b5934700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0067610 0x7fb2b00b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37246/0 (socket says 172.21.15.7:37246) 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b5934700 1 -- 172.21.15.7:0/2673364935 learned_addr learned my addr 172.21.15.7:0/2673364935 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b6135700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b0067240 0x7fb2b0164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b6936700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0157300 0x7fb2b00b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b5934700 1 -- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0157300 msgr2=0x7fb2b00b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b5934700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0157300 0x7fb2b00b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b5934700 1 -- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b0067240 msgr2=0x7fb2b0164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b5934700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b0067240 0x7fb2b0164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b5934700 1 -- 172.21.15.7:0/2673364935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac009420 con 0x7fb2b0067610 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b6936700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0157300 0x7fb2b00b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.424+0000 7fb2b6135700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b0067240 0x7fb2b0164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.425+0000 7fb2b5934700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0067610 0x7fb2b00b0d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00c3f30 tx=0x7fb2ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.425+0000 7fb2a67fc700 1 -- 172.21.15.7:0/2673364935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00bba0 con 0x7fb2b0067610 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.425+0000 7fb2a67fc700 1 -- 172.21.15.7:0/2673364935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac025420 con 0x7fb2b0067610 2023-10-07T07:59:38.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.425+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b00ae8f0 con 0x7fb2b0067610 2023-10-07T07:59:38.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.425+0000 7fb2a67fc700 1 -- 172.21.15.7:0/2673364935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0257e0 con 0x7fb2b0067610 2023-10-07T07:59:38.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.425+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b0167d70 con 0x7fb2b0067610 2023-10-07T07:59:38.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.426+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b004c3c0 con 0x7fb2b0067610 2023-10-07T07:59:38.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.426+0000 7fb2a67fc700 1 -- 172.21.15.7:0/2673364935 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2ac023070 con 0x7fb2b0067610 2023-10-07T07:59:38.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.427+0000 7fb2a67fc700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb294066f30 0x7fb2940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:38.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.427+0000 7fb2a67fc700 1 -- 172.21.15.7:0/2673364935 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5907+0+0 (secure 0 0 0) 0x7fb2ac085e20 con 0x7fb2b0067610 2023-10-07T07:59:38.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.427+0000 7fb2b6135700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb294066f30 0x7fb2940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:38.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.427+0000 7fb2b6135700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb294066f30 0x7fb2940693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00018d0 tx=0x7fb2a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:38.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.431+0000 7fb2a67fc700 1 -- 172.21.15.7:0/2673364935 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2ac052f20 con 0x7fb2b0067610 2023-10-07T07:59:38.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:38.679+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb2b00c79d0 con 0x7fb2b0067610 2023-10-07T07:59:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.313+0000 7fb2a67fc700 1 -- 172.21.15.7:0/2673364935 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb2ac056540 con 0x7fb2b0067610 2023-10-07T07:59:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb294066f30 msgr2=0x7fb2940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb294066f30 0x7fb2940693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00018d0 tx=0x7fb2a0009040 comp rx=0 tx=0).stop 2023-10-07T07:59:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0067610 msgr2=0x7fb2b00b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0067610 0x7fb2b00b0d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00c3f30 tx=0x7fb2ac017000 comp rx=0 tx=0).stop 2023-10-07T07:59:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 shutdown_connections 2023-10-07T07:59:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb294066f30 0x7fb2940693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b0157300 0x7fb2b00b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b0067240 0x7fb2b0164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 --2- 172.21.15.7:0/2673364935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b0067610 0x7fb2b00b0d10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.316+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 >> 172.21.15.7:0/2673364935 conn(0x7fb2b0068100 msgr2=0x7fb2b00c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.317+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 shutdown_connections 2023-10-07T07:59:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.317+0000 7fb2b8399700 1 -- 172.21.15.7:0/2673364935 wait complete. 2023-10-07T07:59:39.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-07T07:59:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.518+0000 7f0c26c9c700 1 Processor -- start 2023-10-07T07:59:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.518+0000 7f0c26c9c700 1 -- start start 2023-10-07T07:59:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.518+0000 7f0c26c9c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c20140bf0 0x7f0c2015b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.518+0000 7f0c26c9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c200ad960 0x7f0c200ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.518+0000 7f0c26c9c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200cc040 con 0x7f0c2006b560 2023-10-07T07:59:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.518+0000 7f0c26c9c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c200addd0 con 0x7f0c200ad960 2023-10-07T07:59:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c26c9c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200cbd10 con 0x7f0c20140bf0 2023-10-07T07:59:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c200ad960 0x7f0c200ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c25239700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0c2006b560 0x7f0c200cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54502/0 (socket says 172.21.15.7:54502) 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c25239700 1 -- 172.21.15.7:0/887141469 learned_addr learned my addr 172.21.15.7:0/887141469 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c24a38700 1 --2- 172.21.15.7:0/887141469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c20140bf0 0x7f0c2015b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1ffff700 1 -- 172.21.15.7:0/887141469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c20140bf0 msgr2=0x7f0c2015b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1ffff700 1 --2- 172.21.15.7:0/887141469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c20140bf0 0x7f0c2015b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1ffff700 1 -- 172.21.15.7:0/887141469 >> v1:172.21.15.7:6789/0 conn(0x7f0c2006b560 legacy=0x7f0c200cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1ffff700 1 -- 172.21.15.7:0/887141469 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200c8780 con 0x7f0c200ad960 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1f7fe700 1 -- 172.21.15.7:0/887141469 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911384427 0 0) 0x7f0c200cc040 con 0x7f0c2006b560 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1ffff700 1 --2- 172.21.15.7:0/887141469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c200ad960 0x7f0c200ca2e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0098e0 tx=0x7f0c0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e685fe2b33fd20 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1f7fe700 1 -- 172.21.15.7:0/887141469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c009e80 con 0x7f0c200ad960 2023-10-07T07:59:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1f7fe700 1 -- 172.21.15.7:0/887141469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c00b860 con 0x7f0c200ad960 2023-10-07T07:59:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.519+0000 7f0c1f7fe700 1 -- 172.21.15.7:0/887141469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00bbe0 con 0x7f0c200ad960 2023-10-07T07:59:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.520+0000 7f0c26c9c700 1 -- 172.21.15.7:0/887141469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c200ad960 msgr2=0x7f0c200ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.520+0000 7f0c26c9c700 1 --2- 172.21.15.7:0/887141469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c200ad960 0x7f0c200ca2e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0098e0 tx=0x7f0c0c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.520+0000 7f0c26c9c700 1 -- 172.21.15.7:0/887141469 shutdown_connections 2023-10-07T07:59:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.520+0000 7f0c26c9c700 1 --2- 172.21.15.7:0/887141469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c20140bf0 0x7f0c2015b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.520+0000 7f0c26c9c700 1 --2- 172.21.15.7:0/887141469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c200ad960 0x7f0c200ca2e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.520+0000 7f0c26c9c700 1 -- 172.21.15.7:0/887141469 >> 172.21.15.7:0/887141469 conn(0x7f0c20067ac0 msgr2=0x7f0c200ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.520+0000 7f0c26c9c700 1 -- 172.21.15.7:0/887141469 shutdown_connections 2023-10-07T07:59:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.520+0000 7f0c26c9c700 1 -- 172.21.15.7:0/887141469 wait complete. 2023-10-07T07:59:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.521+0000 7f0c26c9c700 1 Processor -- start 2023-10-07T07:59:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.522+0000 7f0c26c9c700 1 -- start start 2023-10-07T07:59:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c26c9c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c2006b560 0x7f0c200a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c26c9c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c200ad960 0x7f0c200af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c26c9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20140bf0 0x7f0c200af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c26c9c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2013f560 con 0x7f0c20140bf0 2023-10-07T07:59:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c26c9c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20158260 con 0x7f0c200ad960 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c26c9c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20158540 con 0x7f0c2006b560 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c25239700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20140bf0 0x7f0c200af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c24a38700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c2006b560 0x7f0c200a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c25239700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20140bf0 0x7f0c200af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47502/0 (socket says 172.21.15.7:47502) 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c1ffff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c200ad960 0x7f0c200af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c25239700 1 -- 172.21.15.7:0/4257790383 learned_addr learned my addr 172.21.15.7:0/4257790383 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c25239700 1 -- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c2006b560 msgr2=0x7f0c200a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c25239700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c2006b560 0x7f0c200a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c25239700 1 -- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c200ad960 msgr2=0x7f0c200af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c25239700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c200ad960 0x7f0c200af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c25239700 1 -- 172.21.15.7:0/4257790383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c009420 con 0x7f0c20140bf0 2023-10-07T07:59:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.523+0000 7f0c25239700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20140bf0 0x7f0c200af5d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800cc90 tx=0x7f0c18001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.524+0000 7f0c1cff9700 1 -- 172.21.15.7:0/4257790383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800a8b0 con 0x7f0c20140bf0 2023-10-07T07:59:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.524+0000 7f0c1cff9700 1 -- 172.21.15.7:0/4257790383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1800aa10 con 0x7f0c20140bf0 2023-10-07T07:59:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.524+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c20158cc0 con 0x7f0c20140bf0 2023-10-07T07:59:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.524+0000 7f0c1cff9700 1 -- 172.21.15.7:0/4257790383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800add0 con 0x7f0c20140bf0 2023-10-07T07:59:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.524+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c20159280 con 0x7f0c20140bf0 2023-10-07T07:59:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.525+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c200716c0 con 0x7f0c20140bf0 2023-10-07T07:59:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.525+0000 7f0c1cff9700 1 -- 172.21.15.7:0/4257790383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c18007300 con 0x7f0c20140bf0 2023-10-07T07:59:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.526+0000 7f0c1cff9700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0c04066f30 0x7f0c040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.526+0000 7f0c1cff9700 1 -- 172.21.15.7:0/4257790383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f0c18084920 con 0x7f0c20140bf0 2023-10-07T07:59:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.526+0000 7f0c24a38700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0c04066f30 0x7f0c040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.527+0000 7f0c24a38700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0c04066f30 0x7f0c040693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c140061f0 tx=0x7f0c14008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.530+0000 7f0c1cff9700 1 -- 172.21.15.7:0/4257790383 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c18051b40 con 0x7f0c20140bf0 2023-10-07T07:59:39.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:39.732+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c200c8a60 con 0x7f0c20140bf0 2023-10-07T07:59:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.324+0000 7f0c1cff9700 1 -- 172.21.15.7:0/4257790383 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0c18055160 con 0x7f0c20140bf0 2023-10-07T07:59:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0c04066f30 msgr2=0x7f0c040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0c04066f30 0x7f0c040693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c140061f0 tx=0x7f0c14008040 comp rx=0 tx=0).stop 2023-10-07T07:59:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20140bf0 msgr2=0x7f0c200af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20140bf0 0x7f0c200af5d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800cc90 tx=0x7f0c18001a80 comp rx=0 tx=0).stop 2023-10-07T07:59:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 shutdown_connections 2023-10-07T07:59:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0c04066f30 0x7f0c040693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20140bf0 0x7f0c200af5d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c2006b560 0x7f0c200a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 --2- 172.21.15.7:0/4257790383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c200ad960 0x7f0c200af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 >> 172.21.15.7:0/4257790383 conn(0x7f0c20067ac0 msgr2=0x7f0c2013fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 shutdown_connections 2023-10-07T07:59:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.327+0000 7f0c26c9c700 1 -- 172.21.15.7:0/4257790383 wait complete. 2023-10-07T07:59:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2023-10-07T07:59:40.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-07T07:59:40.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.483+0000 7f0f94ffd700 1 Processor -- start 2023-10-07T07:59:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.483+0000 7f0f94ffd700 1 -- start start 2023-10-07T07:59:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.484+0000 7f0f94ffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 0x7f0f900afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.484+0000 7f0f94ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f900c9f20 0x7f0f900acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.484+0000 7f0f94ffd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f9015ab50 con 0x7f0f900ac710 2023-10-07T07:59:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.484+0000 7f0f94ffd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f900b0280 con 0x7f0f900c9f20 2023-10-07T07:59:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.484+0000 7f0f94ffd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9006b160 con 0x7f0f9015a780 2023-10-07T07:59:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.484+0000 7f0f8ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f900c9f20 0x7f0f900acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.484+0000 7f0f85d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0f900ac710 0x7f0f900c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54510/0 (socket says 172.21.15.7:54510) 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.484+0000 7f0f85d9b700 1 -- 172.21.15.7:0/1618642872 learned_addr learned my addr 172.21.15.7:0/1618642872 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.485+0000 7f0f8e59c700 1 --2- 172.21.15.7:0/1618642872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 0x7f0f900afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.485+0000 7f0f8ed9d700 1 -- 172.21.15.7:0/1618642872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 msgr2=0x7f0f900afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.485+0000 7f0f8ed9d700 1 --2- 172.21.15.7:0/1618642872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 0x7f0f900afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.485+0000 7f0f8ed9d700 1 -- 172.21.15.7:0/1618642872 >> v1:172.21.15.7:6789/0 conn(0x7f0f900ac710 legacy=0x7f0f900c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.485+0000 7f0f8ed9d700 1 -- 172.21.15.7:0/1618642872 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9006b440 con 0x7f0f900c9f20 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.485+0000 7f0f8ed9d700 1 --2- 172.21.15.7:0/1618642872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 0x7f0f900acb80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009e60 tx=0x7f0f88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3076c0a4c6e8d3b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.485+0000 7f0f8dd9b700 1 -- 172.21.15.7:0/1618642872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800e070 con 0x7f0f900c9f20 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.485+0000 7f0f8dd9b700 1 -- 172.21.15.7:0/1618642872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8800bb30 con 0x7f0f900c9f20 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.485+0000 7f0f8dd9b700 1 -- 172.21.15.7:0/1618642872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88012430 con 0x7f0f900c9f20 2023-10-07T07:59:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.486+0000 7f0f94ffd700 1 -- 172.21.15.7:0/1618642872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 msgr2=0x7f0f900acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.486+0000 7f0f94ffd700 1 --2- 172.21.15.7:0/1618642872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 0x7f0f900acb80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009e60 tx=0x7f0f88009780 comp rx=0 tx=0).stop 2023-10-07T07:59:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.486+0000 7f0f94ffd700 1 -- 172.21.15.7:0/1618642872 shutdown_connections 2023-10-07T07:59:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.486+0000 7f0f94ffd700 1 --2- 172.21.15.7:0/1618642872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 0x7f0f900acb80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.486+0000 7f0f94ffd700 1 --2- 172.21.15.7:0/1618642872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 0x7f0f900afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.486+0000 7f0f94ffd700 1 -- 172.21.15.7:0/1618642872 >> 172.21.15.7:0/1618642872 conn(0x7f0f90068100 msgr2=0x7f0f90068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.486+0000 7f0f94ffd700 1 -- 172.21.15.7:0/1618642872 shutdown_connections 2023-10-07T07:59:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.486+0000 7f0f94ffd700 1 -- 172.21.15.7:0/1618642872 wait complete. 2023-10-07T07:59:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.488+0000 7f0f94ffd700 1 Processor -- start 2023-10-07T07:59:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.488+0000 7f0f94ffd700 1 -- start start 2023-10-07T07:59:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f94ffd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f900ac710 0x7f0f900b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f94ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 0x7f0f901557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f94ffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 0x7f0f90153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f94ffd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9015b6d0 con 0x7f0f900c9f20 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f94ffd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9015b860 con 0x7f0f900ac710 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f94ffd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900b0b10 con 0x7f0f9015a780 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f85d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 0x7f0f901557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f85d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 0x7f0f901557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47522/0 (socket says 172.21.15.7:47522) 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f85d9b700 1 -- 172.21.15.7:0/3231093991 learned_addr learned my addr 172.21.15.7:0/3231093991 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f8ed9d700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 0x7f0f90153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f8e59c700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f900ac710 0x7f0f900b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f85d9b700 1 -- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 msgr2=0x7f0f90153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f85d9b700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 0x7f0f90153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f85d9b700 1 -- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f900ac710 msgr2=0x7f0f900b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f85d9b700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f900ac710 0x7f0f900b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f85d9b700 1 -- 172.21.15.7:0/3231093991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88009a20 con 0x7f0f900c9f20 2023-10-07T07:59:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f85d9b700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 0x7f0f901557b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c960 tx=0x7f0f78007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.489+0000 7f0f86ffd700 1 -- 172.21.15.7:0/3231093991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78005b30 con 0x7f0f900c9f20 2023-10-07T07:59:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.490+0000 7f0f86ffd700 1 -- 172.21.15.7:0/3231093991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f78005c90 con 0x7f0f900c9f20 2023-10-07T07:59:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.490+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f900b0df0 con 0x7f0f900c9f20 2023-10-07T07:59:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.490+0000 7f0f86ffd700 1 -- 172.21.15.7:0/3231093991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78007340 con 0x7f0f900c9f20 2023-10-07T07:59:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.490+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f900b13b0 con 0x7f0f900c9f20 2023-10-07T07:59:40.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.491+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f900716c0 con 0x7f0f900c9f20 2023-10-07T07:59:40.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.491+0000 7f0f86ffd700 1 -- 172.21.15.7:0/3231093991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f7801b400 con 0x7f0f900c9f20 2023-10-07T07:59:40.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.491+0000 7f0f86ffd700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0f70066f30 0x7f0f700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:40.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.492+0000 7f0f86ffd700 1 -- 172.21.15.7:0/3231093991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5469+0+0 (secure 0 0 0) 0x7f0f78083f70 con 0x7f0f900c9f20 2023-10-07T07:59:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.492+0000 7f0f8e59c700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0f70066f30 0x7f0f700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.492+0000 7f0f8e59c700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0f70066f30 0x7f0f700693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f80006fd0 tx=0x7f0f80008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.496+0000 7f0f86ffd700 1 -- 172.21.15.7:0/3231093991 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f78051360 con 0x7f0f900c9f20 2023-10-07T07:59:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:40.697+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f9015ab50 con 0x7f0f900c9f20 2023-10-07T07:59:41.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.329+0000 7f0f86ffd700 1 -- 172.21.15.7:0/3231093991 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0f9015ab50 con 0x7f0f900c9f20 2023-10-07T07:59:41.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0f70066f30 msgr2=0x7f0f700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0f70066f30 0x7f0f700693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f80006fd0 tx=0x7f0f80008040 comp rx=0 tx=0).stop 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 msgr2=0x7f0f901557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 0x7f0f901557b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c960 tx=0x7f0f78007f30 comp rx=0 tx=0).stop 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 shutdown_connections 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0f70066f30 0x7f0f700693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9015a780 0x7f0f90153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f900ac710 0x7f0f900b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 --2- 172.21.15.7:0/3231093991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f900c9f20 0x7f0f901557b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.333+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 >> 172.21.15.7:0/3231093991 conn(0x7f0f90068100 msgr2=0x7f0f9006ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.334+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 shutdown_connections 2023-10-07T07:59:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.334+0000 7f0f94ffd700 1 -- 172.21.15.7:0/3231093991 wait complete. 2023-10-07T07:59:41.336 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2023-10-07T07:59:41.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2023-10-07T07:59:41.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.491+0000 7fb022eb1700 1 Processor -- start 2023-10-07T07:59:41.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.491+0000 7fb022eb1700 1 -- start start 2023-10-07T07:59:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb022eb1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c15c920 0x7fb01c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb022eb1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb01c15ccf0 0x7fb01c06c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb022eb1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c06c600 con 0x7fb01c0c6d70 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb022eb1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c0cb3c0 con 0x7fb01c15ccf0 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb022eb1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0cab00 con 0x7fb01c15c920 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb02144e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb01c0c6d70 0x7fb01c068000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54520/0 (socket says 172.21.15.7:54520) 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb02144e700 1 -- 172.21.15.7:0/2355256406 learned_addr learned my addr 172.21.15.7:0/2355256406 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb01bfff700 1 --2- 172.21.15.7:0/2355256406 >> v2:172.21.15.172:3300/0 conn(0x7fb01c15ccf0 0x7fb01c06c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb020c4d700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c15c920 0x7fb01c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.492+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186732446 0 0) 0x7fb014001490 con 0x7fb01c0c6d70 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb00c003570 con 0x7fb01c0c6d70 2023-10-07T07:59:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3278102616 0 0) 0x7fb00c003570 con 0x7fb01c0c6d70 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c15c920 msgr2=0x7fb01c0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c15c920 0x7fb01c0ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 >> v2:172.21.15.172:3300/0 conn(0x7fb01c15ccf0 msgr2=0x7fb01c06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 --2- 172.21.15.7:0/2355256406 >> v2:172.21.15.172:3300/0 conn(0x7fb01c15ccf0 0x7fb01c06c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c06a7a0 con 0x7fb01c0c6d70 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7fb014001980 con 0x7fb01c0c6d70 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 >> v1:172.21.15.7:6789/0 conn(0x7fb01c0c6d70 legacy=0x7fb01c068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c003a60 0x7fb00c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb00c004810 0x7fb00c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c15c920 0x7fb00c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0cb3c0 con 0x7fb00c004810 2023-10-07T07:59:41.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0cab00 con 0x7fb00c003a60 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c00bae0 con 0x7fb01c15c920 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01bfff700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb00c004810 0x7fb00c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb020c4d700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c003a60 0x7fb00c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb02144e700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c15c920 0x7fb00c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01bfff700 1 -- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c15c920 msgr2=0x7fb00c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01bfff700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c15c920 0x7fb00c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01bfff700 1 -- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c003a60 msgr2=0x7fb00c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01bfff700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c003a60 0x7fb00c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.493+0000 7fb01bfff700 1 -- 172.21.15.7:0/2355256406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c003570 con 0x7fb00c004810 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb01bfff700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb00c004810 0x7fb00c004c80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb01c15d140 tx=0x7fb0080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01b43eebba86674 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:41.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00800e710 con 0x7fb00c004810 2023-10-07T07:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00800e870 con 0x7fb00c004810 2023-10-07T07:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb01b7fe700 1 -- 172.21.15.7:0/2355256406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00800ec30 con 0x7fb00c004810 2023-10-07T07:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb022eb1700 1 -- 172.21.15.7:0/2355256406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb00c004810 msgr2=0x7fb00c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb022eb1700 1 --2- 172.21.15.7:0/2355256406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb00c004810 0x7fb00c004c80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb01c15d140 tx=0x7fb0080091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb02144e700 1 -- 172.21.15.7:0/2355256406 reap_dead start 2023-10-07T07:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb022eb1700 1 -- 172.21.15.7:0/2355256406 shutdown_connections 2023-10-07T07:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb022eb1700 1 -- 172.21.15.7:0/2355256406 >> 172.21.15.7:0/2355256406 conn(0x7fb01c0c9c30 msgr2=0x7fb01c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb022eb1700 1 -- 172.21.15.7:0/2355256406 shutdown_connections 2023-10-07T07:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.494+0000 7fb022eb1700 1 -- 172.21.15.7:0/2355256406 wait complete. 2023-10-07T07:59:41.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.495+0000 7fb022eb1700 1 Processor -- start 2023-10-07T07:59:41.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.496+0000 7fb022eb1700 1 -- start start 2023-10-07T07:59:41.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.496+0000 7fb022eb1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c004810 0x7fb01c16a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.496+0000 7fb022eb1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c0c5e70 0x7fb01c0b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.496+0000 7fb022eb1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01c0b13b0 0x7fb01c154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.496+0000 7fb022eb1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0c65e0 con 0x7fb01c0b13b0 2023-10-07T07:59:41.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.496+0000 7fb022eb1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0afbc0 con 0x7fb00c004810 2023-10-07T07:59:41.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.496+0000 7fb022eb1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b0590 con 0x7fb01c0c5e70 2023-10-07T07:59:41.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb02144e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01c0b13b0 0x7fb01c154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb02144e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01c0b13b0 0x7fb01c154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47534/0 (socket says 172.21.15.7:47534) 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb02144e700 1 -- 172.21.15.7:0/2043315173 learned_addr learned my addr 172.21.15.7:0/2043315173 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb020c4d700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c004810 0x7fb01c16a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb01bfff700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c0c5e70 0x7fb01c0b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb02144e700 1 -- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c0c5e70 msgr2=0x7fb01c0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb02144e700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c0c5e70 0x7fb01c0b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb02144e700 1 -- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c004810 msgr2=0x7fb01c16a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb02144e700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c004810 0x7fb01c16a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb02144e700 1 -- 172.21.15.7:0/2043315173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c0c6300 con 0x7fb01c0b13b0 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb020c4d700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c004810 0x7fb01c16a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb02144e700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01c0b13b0 0x7fb01c154430 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb01c06ecc0 tx=0x7fb01c06a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb018ff9700 1 -- 172.21.15.7:0/2043315173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014007af0 con 0x7fb01c0b13b0 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb018ff9700 1 -- 172.21.15.7:0/2043315173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0140133f0 con 0x7fb01c0b13b0 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb018ff9700 1 -- 172.21.15.7:0/2043315173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140137b0 con 0x7fb01c0b13b0 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c0af570 con 0x7fb01c0b13b0 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.497+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c0b1ee0 con 0x7fb01c0b13b0 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.498+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c0b20d0 con 0x7fb01c0b13b0 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.499+0000 7fb018ff9700 1 -- 172.21.15.7:0/2043315173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb014013be0 con 0x7fb01c0b13b0 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.499+0000 7fb018ff9700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb000066f30 0x7fb0000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.499+0000 7fb018ff9700 1 -- 172.21.15.7:0/2043315173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb014084a10 con 0x7fb01c0b13b0 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.500+0000 7fb020c4d700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb000066f30 0x7fb0000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.500+0000 7fb020c4d700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb000066f30 0x7fb0000693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb010009390 tx=0x7fb010008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:41.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.504+0000 7fb018ff9700 1 -- 172.21.15.7:0/2043315173 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb014051ee0 con 0x7fb01c0b13b0 2023-10-07T07:59:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.709+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) v1 -- 0x7fb01c06c600 con 0x7fb01c0b13b0 2023-10-07T07:59:41.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.713+0000 7fb018ff9700 1 -- 172.21.15.7:0/2043315173 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v13) v1 ==== 106+0+0 (secure 0 0 0) 0x7fb01c06c600 con 0x7fb01c0b13b0 2023-10-07T07:59:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb000066f30 msgr2=0x7fb0000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb000066f30 0x7fb0000693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb010009390 tx=0x7fb010008040 comp rx=0 tx=0).stop 2023-10-07T07:59:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01c0b13b0 msgr2=0x7fb01c154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01c0b13b0 0x7fb01c154430 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb01c06ecc0 tx=0x7fb01c06a710 comp rx=0 tx=0).stop 2023-10-07T07:59:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 shutdown_connections 2023-10-07T07:59:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb000066f30 0x7fb0000693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01c0b13b0 0x7fb01c154430 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00c004810 0x7fb01c16a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 --2- 172.21.15.7:0/2043315173 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c0c5e70 0x7fb01c0b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.715+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 >> 172.21.15.7:0/2043315173 conn(0x7fb01c0c9c30 msgr2=0x7fb01c06db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.716+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 shutdown_connections 2023-10-07T07:59:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.716+0000 7fb022eb1700 1 -- 172.21.15.7:0/2043315173 wait complete. 2023-10-07T07:59:41.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_cap_acquisition_throttle 2023-10-07T07:59:41.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.863+0000 7ff9180ca700 1 Processor -- start 2023-10-07T07:59:41.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.863+0000 7ff9180ca700 1 -- start start 2023-10-07T07:59:41.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.864+0000 7ff9180ca700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff910067700 0x7ff91015e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.864+0000 7ff9180ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff91006e3c0 0x7ff9100cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.864+0000 7ff9180ca700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9100cccb0 con 0x7ff910147030 2023-10-07T07:59:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.864+0000 7ff9180ca700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91006a500 con 0x7ff910067700 2023-10-07T07:59:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.864+0000 7ff9180ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9100cae60 con 0x7ff91006e3c0 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.864+0000 7ff915665700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff910067700 0x7ff91015e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff915665700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff910067700 0x7ff91015e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47538/0 (socket says 172.21.15.7:47538) 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff915665700 1 -- 172.21.15.7:0/2789683104 learned_addr learned my addr 172.21.15.7:0/2789683104 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff916667700 1 --2- 172.21.15.7:0/2789683104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff91006e3c0 0x7ff9100cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff915665700 1 -- 172.21.15.7:0/2789683104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff91006e3c0 msgr2=0x7ff9100cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff915665700 1 --2- 172.21.15.7:0/2789683104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff91006e3c0 0x7ff9100cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff915665700 1 -- 172.21.15.7:0/2789683104 >> v1:172.21.15.172:6789/0 conn(0x7ff910147030 legacy=0x7ff910066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff915665700 1 -- 172.21.15.7:0/2789683104 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91015eca0 con 0x7ff910067700 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff915665700 1 --2- 172.21.15.7:0/2789683104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 0x7ff91015e760 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff91006a640 tx=0x7ff900009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd55b3b595bee403 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff914e64700 1 -- 172.21.15.7:0/2789683104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90000b860 con 0x7ff910067700 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff914e64700 1 -- 172.21.15.7:0/2789683104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90000b9c0 con 0x7ff910067700 2023-10-07T07:59:41.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.865+0000 7ff914e64700 1 -- 172.21.15.7:0/2789683104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90000bd80 con 0x7ff910067700 2023-10-07T07:59:41.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.866+0000 7ff9180ca700 1 -- 172.21.15.7:0/2789683104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 msgr2=0x7ff91015e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.866+0000 7ff9180ca700 1 --2- 172.21.15.7:0/2789683104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 0x7ff91015e760 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff91006a640 tx=0x7ff900009250 comp rx=0 tx=0).stop 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.866+0000 7ff9180ca700 1 -- 172.21.15.7:0/2789683104 shutdown_connections 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.866+0000 7ff9180ca700 1 --2- 172.21.15.7:0/2789683104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff91006e3c0 0x7ff9100cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.866+0000 7ff9180ca700 1 --2- 172.21.15.7:0/2789683104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 0x7ff91015e760 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.866+0000 7ff9180ca700 1 -- 172.21.15.7:0/2789683104 >> 172.21.15.7:0/2789683104 conn(0x7ff910068100 msgr2=0x7ff910068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.866+0000 7ff9180ca700 1 -- 172.21.15.7:0/2789683104 shutdown_connections 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.866+0000 7ff9180ca700 1 -- 172.21.15.7:0/2789683104 wait complete. 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.868+0000 7ff9180ca700 1 Processor -- start 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.868+0000 7ff9180ca700 1 -- start start 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff9180ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 0x7ff9100c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff9180ca700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e3c0 0x7ff91006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff9180ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910147030 0x7ff910144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff9180ca700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9100b0000 con 0x7ff910067700 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff9180ca700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9100b0b40 con 0x7ff91006e3c0 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff9180ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9100c6ff0 con 0x7ff910147030 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915e66700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 0x7ff9100c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915e66700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 0x7ff9100c6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47540/0 (socket says 172.21.15.7:47540) 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915e66700 1 -- 172.21.15.7:0/758371880 learned_addr learned my addr 172.21.15.7:0/758371880 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915665700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e3c0 0x7ff91006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff916667700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910147030 0x7ff910144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915e66700 1 -- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910147030 msgr2=0x7ff910144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915e66700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910147030 0x7ff910144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915e66700 1 -- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e3c0 msgr2=0x7ff91006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915e66700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e3c0 0x7ff91006ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915e66700 1 -- 172.21.15.7:0/758371880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000094f0 con 0x7ff910067700 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff916667700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910147030 0x7ff910144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff915e66700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 0x7ff9100c6ab0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c590 tx=0x7ff90c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff905ffb700 1 -- 172.21.15.7:0/758371880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c011590 con 0x7ff910067700 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff905ffb700 1 -- 172.21.15.7:0/758371880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c011730 con 0x7ff910067700 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9100b0fd0 con 0x7ff910067700 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.869+0000 7ff905ffb700 1 -- 172.21.15.7:0/758371880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c011af0 con 0x7ff910067700 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.870+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9100b04f0 con 0x7ff910067700 2023-10-07T07:59:41.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.871+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9100b0e20 con 0x7ff910067700 2023-10-07T07:59:41.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.871+0000 7ff905ffb700 1 -- 172.21.15.7:0/758371880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff90c012420 con 0x7ff910067700 2023-10-07T07:59:41.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.871+0000 7ff905ffb700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff8f4066f30 0x7ff8f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:41.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.871+0000 7ff905ffb700 1 -- 172.21.15.7:0/758371880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff90c0834d0 con 0x7ff910067700 2023-10-07T07:59:41.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.872+0000 7ff915665700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff8f4066f30 0x7ff8f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:41.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.872+0000 7ff915665700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff8f4066f30 0x7ff8f40693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff91006a640 tx=0x7ff900016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:41.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:41.876+0000 7ff905ffb700 1 -- 172.21.15.7:0/758371880 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff90c04b5a0 con 0x7ff910067700 2023-10-07T07:59:42.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.081+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"} v 0) v1 -- 0x7ff91004c3c0 con 0x7ff910067700 2023-10-07T07:59:42.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.085+0000 7ff905ffb700 1 -- 172.21.15.7:0/758371880 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"}]=0 v14) v1 ==== 119+0+0 (secure 0 0 0) 0x7ff90c017020 con 0x7ff910067700 2023-10-07T07:59:42.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.087+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff8f4066f30 msgr2=0x7ff8f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.087+0000 7ff9180ca700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff8f4066f30 0x7ff8f40693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff91006a640 tx=0x7ff900016000 comp rx=0 tx=0).stop 2023-10-07T07:59:42.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 msgr2=0x7ff9100c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 0x7ff9100c6ab0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c590 tx=0x7ff90c007f30 comp rx=0 tx=0).stop 2023-10-07T07:59:42.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 shutdown_connections 2023-10-07T07:59:42.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff8f4066f30 0x7ff8f40693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910147030 0x7ff910144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910067700 0x7ff9100c6ab0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 --2- 172.21.15.7:0/758371880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e3c0 0x7ff91006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 >> 172.21.15.7:0/758371880 conn(0x7ff910068100 msgr2=0x7ff9100cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:42.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 shutdown_connections 2023-10-07T07:59:42.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.088+0000 7ff9180ca700 1 -- 172.21.15.7:0/758371880 wait complete. 2023-10-07T07:59:42.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-07T07:59:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.236+0000 7fb30d39a700 1 Processor -- start 2023-10-07T07:59:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.236+0000 7fb30d39a700 1 -- start start 2023-10-07T07:59:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.237+0000 7fb30d39a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb3080af810 0x7fb3080ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.237+0000 7fb30d39a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3080cf1e0 con 0x7fb30815dbf0 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.237+0000 7fb30d39a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3080ce5d0 con 0x7fb3080af440 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.237+0000 7fb30d39a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3080ce8b0 con 0x7fb3080af810 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.237+0000 7fb3067fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb30815dbf0 0x7fb3080afd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54522/0 (socket says 172.21.15.7:54522) 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.237+0000 7fb3067fc700 1 -- 172.21.15.7:0/1782175980 learned_addr learned my addr 172.21.15.7:0/1782175980 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb3077fe700 1 --2- 172.21.15.7:0/1782175980 >> v2:172.21.15.172:3300/0 conn(0x7fb3080af810 0x7fb3080ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb305ffb700 1 -- 172.21.15.7:0/1782175980 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80023623 0 0) 0x7fb3080cf1e0 con 0x7fb30815dbf0 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb305ffb700 1 -- 172.21.15.7:0/1782175980 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2f4003570 con 0x7fb30815dbf0 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb305ffb700 1 -- 172.21.15.7:0/1782175980 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792555205 0 0) 0x7fb3080ce5d0 con 0x7fb3080af440 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb305ffb700 1 -- 172.21.15.7:0/1782175980 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3080cf1e0 con 0x7fb3080af440 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb3077fe700 1 -- 172.21.15.7:0/1782175980 >> v1:172.21.15.172:6789/0 conn(0x7fb3080af440 legacy=0x7fb3080cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb3077fe700 1 -- 172.21.15.7:0/1782175980 >> v1:172.21.15.7:6789/0 conn(0x7fb30815dbf0 legacy=0x7fb3080afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb3077fe700 1 -- 172.21.15.7:0/1782175980 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3080ceb90 con 0x7fb3080af810 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb3077fe700 1 --2- 172.21.15.7:0/1782175980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3080af810 0x7fb3080ccc70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb300009e60 tx=0x7fb300009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5638dc838548678d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb305ffb700 1 -- 172.21.15.7:0/1782175980 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30000e070 con 0x7fb3080af810 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb305ffb700 1 -- 172.21.15.7:0/1782175980 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30000bb40 con 0x7fb3080af810 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.238+0000 7fb305ffb700 1 -- 172.21.15.7:0/1782175980 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300012430 con 0x7fb3080af810 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.239+0000 7fb30d39a700 1 -- 172.21.15.7:0/1782175980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3080af810 msgr2=0x7fb3080ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.239+0000 7fb30d39a700 1 --2- 172.21.15.7:0/1782175980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3080af810 0x7fb3080ccc70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb300009e60 tx=0x7fb300009780 comp rx=0 tx=0).stop 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.239+0000 7fb30d39a700 1 -- 172.21.15.7:0/1782175980 shutdown_connections 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.239+0000 7fb30d39a700 1 --2- 172.21.15.7:0/1782175980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3080af810 0x7fb3080ccc70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.239+0000 7fb30d39a700 1 -- 172.21.15.7:0/1782175980 >> 172.21.15.7:0/1782175980 conn(0x7fb308069220 msgr2=0x7fb308069620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.239+0000 7fb30d39a700 1 -- 172.21.15.7:0/1782175980 shutdown_connections 2023-10-07T07:59:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.239+0000 7fb30d39a700 1 -- 172.21.15.7:0/1782175980 wait complete. 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.240+0000 7fb30d39a700 1 Processor -- start 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.240+0000 7fb30d39a700 1 -- start start 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.241+0000 7fb30d39a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3080af440 0x7fb3080c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.241+0000 7fb30d39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3080af810 0x7fb308169490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.241+0000 7fb30d39a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb30815dbf0 0x7fb308147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.241+0000 7fb30d39a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb308169b20 con 0x7fb3080af810 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.241+0000 7fb30d39a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb308169e00 con 0x7fb30815dbf0 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.241+0000 7fb30d39a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30816a0e0 con 0x7fb3080af440 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.241+0000 7fb3067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3080af810 0x7fb308169490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.241+0000 7fb3067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3080af810 0x7fb308169490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47548/0 (socket says 172.21.15.7:47548) 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.241+0000 7fb3067fc700 1 -- 172.21.15.7:0/2667323430 learned_addr learned my addr 172.21.15.7:0/2667323430 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb306ffd700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3080af440 0x7fb3080c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb3077fe700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb30815dbf0 0x7fb308147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb3067fc700 1 -- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3080af440 msgr2=0x7fb3080c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb3067fc700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3080af440 0x7fb3080c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb3067fc700 1 -- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb30815dbf0 msgr2=0x7fb308147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb3067fc700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb30815dbf0 0x7fb308147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb3067fc700 1 -- 172.21.15.7:0/2667323430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300009a20 con 0x7fb3080af810 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb3077fe700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb30815dbf0 0x7fb308147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb3067fc700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3080af810 0x7fb308169490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800c790 tx=0x7fb2f8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:42.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb2feffd700 1 -- 172.21.15.7:0/2667323430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8005c00 con 0x7fb3080af810 2023-10-07T07:59:42.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb2feffd700 1 -- 172.21.15.7:0/2667323430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f8005d60 con 0x7fb3080af810 2023-10-07T07:59:42.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb30816a3c0 con 0x7fb3080af810 2023-10-07T07:59:42.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb2feffd700 1 -- 172.21.15.7:0/2667323430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8007300 con 0x7fb3080af810 2023-10-07T07:59:42.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.242+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3081534d0 con 0x7fb3080af810 2023-10-07T07:59:42.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.243+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb30804c3c0 con 0x7fb3080af810 2023-10-07T07:59:42.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.244+0000 7fb2feffd700 1 -- 172.21.15.7:0/2667323430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2f801b3b0 con 0x7fb3080af810 2023-10-07T07:59:42.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.245+0000 7fb2feffd700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb2e8066f30 0x7fb2e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.245+0000 7fb2feffd700 1 -- 172.21.15.7:0/2667323430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb2f8083c20 con 0x7fb3080af810 2023-10-07T07:59:42.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.245+0000 7fb306ffd700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb2e8066f30 0x7fb2e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.246+0000 7fb306ffd700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb2e8066f30 0x7fb2e80693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0007fd0 tx=0x7fb2f0007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:42.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.249+0000 7fb2feffd700 1 -- 172.21.15.7:0/2667323430 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2f8051210 con 0x7fb3080af810 2023-10-07T07:59:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.451+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb3080cdfc0 con 0x7fb3080af810 2023-10-07T07:59:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.455+0000 7fb2feffd700 1 -- 172.21.15.7:0/2667323430 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb2f8058e60 con 0x7fb3080af810 2023-10-07T07:59:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb2e8066f30 msgr2=0x7fb2e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb2e8066f30 0x7fb2e80693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0007fd0 tx=0x7fb2f0007d50 comp rx=0 tx=0).stop 2023-10-07T07:59:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3080af810 msgr2=0x7fb308169490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3080af810 0x7fb308169490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800c790 tx=0x7fb2f8007ea0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 shutdown_connections 2023-10-07T07:59:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb2e8066f30 0x7fb2e80693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb30815dbf0 0x7fb308147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3080af440 0x7fb3080c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 --2- 172.21.15.7:0/2667323430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3080af810 0x7fb308169490 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 >> 172.21.15.7:0/2667323430 conn(0x7fb308069220 msgr2=0x7fb308153f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 shutdown_connections 2023-10-07T07:59:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.457+0000 7fb30d39a700 1 -- 172.21.15.7:0/2667323430 wait complete. 2023-10-07T07:59:42.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-10-07T07:59:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.603+0000 7f2514d6a700 1 Processor -- start 2023-10-07T07:59:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.603+0000 7f2514d6a700 1 -- start start 2023-10-07T07:59:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.604+0000 7f2514d6a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2510159690 0x7f25100cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.604+0000 7f2514d6a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251006bb70 con 0x7f25101592c0 2023-10-07T07:59:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.604+0000 7f2514d6a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25100cda10 con 0x7f25100cb0c0 2023-10-07T07:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.604+0000 7f2514d6a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251006af60 con 0x7f2510159690 2023-10-07T07:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.604+0000 7f250e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f25101592c0 0x7f251006a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54530/0 (socket says 172.21.15.7:54530) 2023-10-07T07:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.604+0000 7f250e59c700 1 -- 172.21.15.7:0/3154188478 learned_addr learned my addr 172.21.15.7:0/3154188478 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.604+0000 7f2507fff700 1 --2- 172.21.15.7:0/3154188478 >> v2:172.21.15.172:3300/0 conn(0x7f2510159690 0x7f25100cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343591071 0 0) 0x7f251006bb70 con 0x7f25101592c0 2023-10-07T07:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24fc003570 con 0x7f25101592c0 2023-10-07T07:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2869892830 0 0) 0x7f24fc003570 con 0x7f25101592c0 2023-10-07T07:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 >> v2:172.21.15.172:3300/0 conn(0x7f2510159690 msgr2=0x7f25100cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 --2- 172.21.15.7:0/3154188478 >> v2:172.21.15.172:3300/0 conn(0x7f2510159690 0x7f25100cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 >> v1:172.21.15.172:6789/0 conn(0x7f25100cb0c0 legacy=0x7f25100a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251006b240 con 0x7f25101592c0 2023-10-07T07:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f2500001470 con 0x7f25101592c0 2023-10-07T07:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 >> v1:172.21.15.7:6789/0 conn(0x7f25101592c0 legacy=0x7f251006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24fc003a60 0x7f24fc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24fc004810 0x7f24fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc0051c0 0x7f24fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251006af60 con 0x7f24fc003a60 2023-10-07T07:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc00c130 con 0x7f24fc0051c0 2023-10-07T07:59:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc00c2f0 con 0x7f24fc004810 2023-10-07T07:59:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250e59c700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24fc003a60 0x7f24fc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f2507fff700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24fc004810 0x7f24fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250ed9d700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc0051c0 0x7f24fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250e59c700 1 -- 172.21.15.7:0/3154188478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24fc004810 msgr2=0x7f24fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250e59c700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24fc004810 0x7f24fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250e59c700 1 -- 172.21.15.7:0/3154188478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc0051c0 msgr2=0x7f24fc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250e59c700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc0051c0 0x7f24fc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250e59c700 1 -- 172.21.15.7:0/3154188478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc003570 con 0x7f24fc003a60 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250ed9d700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc0051c0 0x7f24fc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.605+0000 7f250ed9d700 1 -- 172.21.15.7:0/3154188478 reap_dead start 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f250e59c700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24fc003a60 0x7f24fc003e30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f250000c2a0 tx=0x7f2500001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea109a1c27b0fe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250000cb70 con 0x7f24fc003a60 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250000ccf0 con 0x7f24fc003a60 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f250dd9b700 1 -- 172.21.15.7:0/3154188478 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500005b70 con 0x7f24fc003a60 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f2514d6a700 1 -- 172.21.15.7:0/3154188478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24fc003a60 msgr2=0x7f24fc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f2514d6a700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24fc003a60 0x7f24fc003e30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f250000c2a0 tx=0x7f2500001da0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f2514d6a700 1 -- 172.21.15.7:0/3154188478 shutdown_connections 2023-10-07T07:59:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f2514d6a700 1 --2- 172.21.15.7:0/3154188478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24fc003a60 0x7f24fc003e30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f2514d6a700 1 -- 172.21.15.7:0/3154188478 >> 172.21.15.7:0/3154188478 conn(0x7f2510068bd0 msgr2=0x7f25100ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f2514d6a700 1 -- 172.21.15.7:0/3154188478 shutdown_connections 2023-10-07T07:59:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.606+0000 7f2514d6a700 1 -- 172.21.15.7:0/3154188478 wait complete. 2023-10-07T07:59:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.608+0000 7f2514d6a700 1 Processor -- start 2023-10-07T07:59:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.608+0000 7f2514d6a700 1 -- start start 2023-10-07T07:59:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.608+0000 7f2514d6a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc003a60 0x7f2510154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.608+0000 7f2514d6a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f251015f110 0x7f251015f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.608+0000 7f2514d6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f251015db20 0x7f251015df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.608+0000 7f2514d6a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251015fac0 con 0x7f251015db20 2023-10-07T07:59:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.608+0000 7f2514d6a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251015a1d0 con 0x7f24fc003a60 2023-10-07T07:59:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.608+0000 7f2514d6a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2510169600 con 0x7f251015f110 2023-10-07T07:59:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.608+0000 7f250ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f251015db20 0x7f251015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f251015db20 0x7f251015df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47572/0 (socket says 172.21.15.7:47572) 2023-10-07T07:59:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250ed9d700 1 -- 172.21.15.7:0/4066935077 learned_addr learned my addr 172.21.15.7:0/4066935077 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250ed9d700 1 -- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f251015f110 msgr2=0x7f251015f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f2507fff700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f251015f110 0x7f251015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250e59c700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc003a60 0x7f2510154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250ed9d700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f251015f110 0x7f251015f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250ed9d700 1 -- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc003a60 msgr2=0x7f2510154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250ed9d700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc003a60 0x7f2510154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250ed9d700 1 -- 172.21.15.7:0/4066935077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251015e8a0 con 0x7f251015db20 2023-10-07T07:59:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f2507fff700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f251015f110 0x7f251015f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250e59c700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc003a60 0x7f2510154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f250ed9d700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f251015db20 0x7f251015df90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f251006dac0 tx=0x7f2508009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f25067fc700 1 -- 172.21.15.7:0/4066935077 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508013630 con 0x7f251015db20 2023-10-07T07:59:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f25067fc700 1 -- 172.21.15.7:0/4066935077 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2508013790 con 0x7f251015db20 2023-10-07T07:59:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251015eb80 con 0x7f251015db20 2023-10-07T07:59:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f25067fc700 1 -- 172.21.15.7:0/4066935077 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508013b50 con 0x7f251015db20 2023-10-07T07:59:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.609+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2510159e60 con 0x7f251015db20 2023-10-07T07:59:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.610+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251015a340 con 0x7f251015db20 2023-10-07T07:59:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.611+0000 7f25067fc700 1 -- 172.21.15.7:0/4066935077 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2508003030 con 0x7f251015db20 2023-10-07T07:59:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.611+0000 7f25067fc700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24f0066f30 0x7f24f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.611+0000 7f25067fc700 1 -- 172.21.15.7:0/4066935077 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f25080840c0 con 0x7f251015db20 2023-10-07T07:59:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.611+0000 7f250e59c700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24f0066f30 0x7f24f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.612+0000 7f250e59c700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24f0066f30 0x7f24f00693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f250000c2a0 tx=0x7f2500015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.616+0000 7f25067fc700 1 -- 172.21.15.7:0/4066935077 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25080550c0 con 0x7f251015db20 2023-10-07T07:59:42.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:42.904+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7f25100716c0 con 0x7f251015db20 2023-10-07T07:59:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.332+0000 7f25067fc700 1 -- 172.21.15.7:0/4066935077 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v105) v1 ==== 165+0+0 (secure 0 0 0) 0x7f2508054c50 con 0x7f251015db20 2023-10-07T07:59:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24f0066f30 msgr2=0x7f24f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24f0066f30 0x7f24f00693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f250000c2a0 tx=0x7f2500015000 comp rx=0 tx=0).stop 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f251015db20 msgr2=0x7f251015df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f251015db20 0x7f251015df90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f251006dac0 tx=0x7f2508009040 comp rx=0 tx=0).stop 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 shutdown_connections 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f24f0066f30 0x7f24f00693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f251015db20 0x7f251015df90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24fc003a60 0x7f2510154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 --2- 172.21.15.7:0/4066935077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f251015f110 0x7f251015f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 >> 172.21.15.7:0/4066935077 conn(0x7f2510068bd0 msgr2=0x7f25100c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.335+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 shutdown_connections 2023-10-07T07:59:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.336+0000 7f2514d6a700 1 -- 172.21.15.7:0/4066935077 wait complete. 2023-10-07T07:59:43.352 INFO:tasks.cephfs_test_runner:Mostly readdir acquires caps faster than the mds recalls, so the cap ... ok 2023-10-07T07:59:43.352 INFO:tasks.cephfs_test_runner:Starting test: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) that a client is blocklisted when its encoded session metadata exceeds the 2023-10-07T07:59:43.353 INFO:tasks.cephfs_test_runner:test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T07:59:43.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2023-10-07T07:59:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.488+0000 7ff11655c700 1 Processor -- start 2023-10-07T07:59:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.488+0000 7ff11655c700 1 -- start start 2023-10-07T07:59:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.489+0000 7ff11655c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff110063230 0x7ff11006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.489+0000 7ff11655c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11015a330 con 0x7ff1100c9af0 2023-10-07T07:59:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.489+0000 7ff11655c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11015a610 con 0x7ff1100c9ec0 2023-10-07T07:59:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.489+0000 7ff11655c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100b03a0 con 0x7ff110063230 2023-10-07T07:59:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.489+0000 7ff10ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff1100c9af0 0x7ff11015d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54538/0 (socket says 172.21.15.7:54538) 2023-10-07T07:59:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.489+0000 7ff10ffff700 1 -- 172.21.15.7:0/2384789746 learned_addr learned my addr 172.21.15.7:0/2384789746 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.489+0000 7ff10f7fe700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff110063230 0x7ff11006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.489+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434980409 0 0) 0x7ff11015a330 con 0x7ff1100c9af0 2023-10-07T07:59:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.489+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0fc003570 con 0x7ff1100c9af0 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 101188310 0 0) 0x7ff0fc003570 con 0x7ff1100c9af0 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff110063230 msgr2=0x7ff11006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff110063230 0x7ff11006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 >> v1:172.21.15.172:6789/0 conn(0x7ff1100c9ec0 legacy=0x7ff11006e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11015a8f0 con 0x7ff1100c9af0 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7ff100001470 con 0x7ff1100c9af0 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 >> v1:172.21.15.7:6789/0 conn(0x7ff1100c9af0 legacy=0x7ff11015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff0fc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff110063230 0x7ff0fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0fc004d50 0x7ff0fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100b03a0 con 0x7ff0fc003a90 2023-10-07T07:59:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc00bb70 con 0x7ff0fc004d50 2023-10-07T07:59:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc00bd30 con 0x7ff110063230 2023-10-07T07:59:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10ffff700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff0fc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10f7fe700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff110063230 0x7ff0fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff114af9700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0fc004d50 0x7ff0fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10ffff700 1 -- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff110063230 msgr2=0x7ff0fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10ffff700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff110063230 0x7ff0fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10ffff700 1 -- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0fc004d50 msgr2=0x7ff0fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10ffff700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0fc004d50 0x7ff0fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10ffff700 1 -- 172.21.15.7:0/2384789746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc003570 con 0x7ff0fc003a90 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.490+0000 7ff10ffff700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff0fc003e60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c2d0 tx=0x7ff100001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3371ea242d0e9d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10000cba0 con 0x7ff0fc003a90 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10000cd20 con 0x7ff0fc003a90 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff10effd700 1 -- 172.21.15.7:0/2384789746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100005b70 con 0x7ff0fc003a90 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff11655c700 1 -- 172.21.15.7:0/2384789746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 msgr2=0x7ff0fc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff11655c700 1 --2- 172.21.15.7:0/2384789746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff0fc003e60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c2d0 tx=0x7ff100001da0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff114af9700 1 -- 172.21.15.7:0/2384789746 reap_dead start 2023-10-07T07:59:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff11655c700 1 -- 172.21.15.7:0/2384789746 shutdown_connections 2023-10-07T07:59:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff11655c700 1 -- 172.21.15.7:0/2384789746 >> 172.21.15.7:0/2384789746 conn(0x7ff1100cd1a0 msgr2=0x7ff1100c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff11655c700 1 -- 172.21.15.7:0/2384789746 shutdown_connections 2023-10-07T07:59:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.491+0000 7ff11655c700 1 -- 172.21.15.7:0/2384789746 wait complete. 2023-10-07T07:59:43.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.493+0000 7ff11655c700 1 Processor -- start 2023-10-07T07:59:43.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.493+0000 7ff11655c700 1 -- start start 2023-10-07T07:59:43.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.494+0000 7ff11655c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff11015b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.494+0000 7ff11655c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff110153fe0 0x7ff110150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.494+0000 7ff11655c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff11006a120 0x7ff11006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.494+0000 7ff11655c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110155cf0 con 0x7ff0fc003a90 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.494+0000 7ff11655c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015baa0 con 0x7ff110153fe0 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.494+0000 7ff11655c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11006aad0 con 0x7ff11006a120 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.494+0000 7ff10ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff11015b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.494+0000 7ff10ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff11015b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47578/0 (socket says 172.21.15.7:47578) 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.494+0000 7ff10ffff700 1 -- 172.21.15.7:0/1750772427 learned_addr learned my addr 172.21.15.7:0/1750772427 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff10f7fe700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff110153fe0 0x7ff110150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff114af9700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff11006a120 0x7ff11006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:43.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff10ffff700 1 -- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff11006a120 msgr2=0x7ff11006a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff10ffff700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff11006a120 0x7ff11006a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff10ffff700 1 -- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff110153fe0 msgr2=0x7ff110150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff10ffff700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff110153fe0 0x7ff110150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff10ffff700 1 -- 172.21.15.7:0/1750772427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff110150840 con 0x7ff0fc003a90 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff114af9700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff11006a120 0x7ff11006a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff10ffff700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff11015b560 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff100001ea0 tx=0x7ff100015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff107fff700 1 -- 172.21.15.7:0/1750772427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000070d0 con 0x7ff0fc003a90 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff107fff700 1 -- 172.21.15.7:0/1750772427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff100007230 con 0x7ff0fc003a90 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff110154550 con 0x7ff0fc003a90 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff107fff700 1 -- 172.21.15.7:0/1750772427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000075f0 con 0x7ff0fc003a90 2023-10-07T07:59:43.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.495+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff110154b10 con 0x7ff0fc003a90 2023-10-07T07:59:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.496+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1100716c0 con 0x7ff0fc003a90 2023-10-07T07:59:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.497+0000 7ff107fff700 1 -- 172.21.15.7:0/1750772427 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff100019030 con 0x7ff0fc003a90 2023-10-07T07:59:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.497+0000 7ff107fff700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff0f0066f30 0x7ff0f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.497+0000 7ff107fff700 1 -- 172.21.15.7:0/1750772427 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff100085360 con 0x7ff0fc003a90 2023-10-07T07:59:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.497+0000 7ff10f7fe700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff0f0066f30 0x7ff0f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.498+0000 7ff10f7fe700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff0f0066f30 0x7ff0f00693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8006dc0 tx=0x7ff0f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.502+0000 7ff107fff700 1 -- 172.21.15.7:0/1750772427 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1000528d0 con 0x7ff0fc003a90 2023-10-07T07:59:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:43.788+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) v1 -- 0x7ff11006ee60 con 0x7ff0fc003a90 2023-10-07T07:59:44.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.339+0000 7ff107fff700 1 -- 172.21.15.7:0/1750772427 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v106) v1 ==== 165+0+0 (secure 0 0 0) 0x7ff100055ef0 con 0x7ff0fc003a90 2023-10-07T07:59:44.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff0f0066f30 msgr2=0x7ff0f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff0f0066f30 0x7ff0f00693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8006dc0 tx=0x7ff0f8008040 comp rx=0 tx=0).stop 2023-10-07T07:59:44.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 msgr2=0x7ff11015b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff11015b560 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff100001ea0 tx=0x7ff100015000 comp rx=0 tx=0).stop 2023-10-07T07:59:44.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 shutdown_connections 2023-10-07T07:59:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff0f0066f30 0x7ff0f00693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff11006a120 0x7ff11006a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0fc003a90 0x7ff11015b560 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 --2- 172.21.15.7:0/1750772427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff110153fe0 0x7ff110150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 >> 172.21.15.7:0/1750772427 conn(0x7ff1100cd1a0 msgr2=0x7ff110067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 shutdown_connections 2023-10-07T07:59:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.342+0000 7ff11655c700 1 -- 172.21.15.7:0/1750772427 wait complete. 2023-10-07T07:59:44.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-07T07:59:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.496+0000 7efe0f634700 1 Processor -- start 2023-10-07T07:59:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.496+0000 7efe0f634700 1 -- start start 2023-10-07T07:59:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.497+0000 7efe0f634700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efe0815d3b0 0x7efe0815d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.497+0000 7efe0f634700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe08069c90 0x7efe0806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.497+0000 7efe0f634700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe080687f0 con 0x7efe0806b610 2023-10-07T07:59:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.497+0000 7efe0f634700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0806bff0 con 0x7efe0815d3b0 2023-10-07T07:59:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.497+0000 7efe0f634700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe08159820 con 0x7efe08069c90 2023-10-07T07:59:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.497+0000 7efe0d3d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efe0815d3b0 0x7efe0815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.497+0000 7efe0cbcf700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efe0806b610 0x7efe0806cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54554/0 (socket says 172.21.15.7:54554) 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.497+0000 7efe0cbcf700 1 -- 172.21.15.7:0/1447607319 learned_addr learned my addr 172.21.15.7:0/1447607319 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.498+0000 7efe0dbd1700 1 --2- 172.21.15.7:0/1447607319 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe08069c90 0x7efe0806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.498+0000 7efe0d3d0700 1 -- 172.21.15.7:0/1447607319 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe08069c90 msgr2=0x7efe0806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.498+0000 7efe0d3d0700 1 --2- 172.21.15.7:0/1447607319 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe08069c90 0x7efe0806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.498+0000 7efe0d3d0700 1 -- 172.21.15.7:0/1447607319 >> v1:172.21.15.7:6789/0 conn(0x7efe0806b610 legacy=0x7efe0806cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.498+0000 7efe0d3d0700 1 -- 172.21.15.7:0/1447607319 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe080c89f0 con 0x7efe0815d3b0 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.498+0000 7efe0d3d0700 1 --2- 172.21.15.7:0/1447607319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 0x7efe0815d780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efe04009910 tx=0x7efe040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeda5565c4d3a8a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.498+0000 7efdfffff700 1 -- 172.21.15.7:0/1447607319 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04009eb0 con 0x7efe0815d3b0 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.498+0000 7efdfffff700 1 -- 172.21.15.7:0/1447607319 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0400b760 con 0x7efe0815d3b0 2023-10-07T07:59:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.498+0000 7efdfffff700 1 -- 172.21.15.7:0/1447607319 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400bb00 con 0x7efe0815d3b0 2023-10-07T07:59:44.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.499+0000 7efe0f634700 1 -- 172.21.15.7:0/1447607319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 msgr2=0x7efe0815d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.499+0000 7efe0f634700 1 --2- 172.21.15.7:0/1447607319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 0x7efe0815d780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efe04009910 tx=0x7efe040091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.499+0000 7efe0f634700 1 -- 172.21.15.7:0/1447607319 shutdown_connections 2023-10-07T07:59:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.499+0000 7efe0f634700 1 --2- 172.21.15.7:0/1447607319 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe08069c90 0x7efe0806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.499+0000 7efe0f634700 1 --2- 172.21.15.7:0/1447607319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 0x7efe0815d780 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.499+0000 7efe0f634700 1 -- 172.21.15.7:0/1447607319 >> 172.21.15.7:0/1447607319 conn(0x7efe08067ac0 msgr2=0x7efe080ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.499+0000 7efe0f634700 1 -- 172.21.15.7:0/1447607319 shutdown_connections 2023-10-07T07:59:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.499+0000 7efe0f634700 1 -- 172.21.15.7:0/1447607319 wait complete. 2023-10-07T07:59:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.500+0000 7efe0f634700 1 Processor -- start 2023-10-07T07:59:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.500+0000 7efe0f634700 1 -- start start 2023-10-07T07:59:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0f634700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe08069c90 0x7efe080af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0f634700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0806b610 0x7efe0816a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0f634700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 0x7efe080caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0f634700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe080af600 con 0x7efe0815d3b0 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0f634700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe08159f60 con 0x7efe08069c90 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0f634700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0815a240 con 0x7efe0806b610 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0dbd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 0x7efe080caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0dbd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 0x7efe080caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47590/0 (socket says 172.21.15.7:47590) 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0dbd1700 1 -- 172.21.15.7:0/2897652132 learned_addr learned my addr 172.21.15.7:0/2897652132 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0d3d0700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe08069c90 0x7efe080af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.501+0000 7efe0cbcf700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0806b610 0x7efe0816a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efe0dbd1700 1 -- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0806b610 msgr2=0x7efe0816a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efe0dbd1700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0806b610 0x7efe0816a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efe0dbd1700 1 -- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe08069c90 msgr2=0x7efe080af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efe0dbd1700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe08069c90 0x7efe080af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efe0dbd1700 1 -- 172.21.15.7:0/2897652132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe04009420 con 0x7efe0815d3b0 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efe0dbd1700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 0x7efe080caae0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efe00009fa0 tx=0x7efe00009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efdfd7fa700 1 -- 172.21.15.7:0/2897652132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0000ebc0 con 0x7efe0815d3b0 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efdfd7fa700 1 -- 172.21.15.7:0/2897652132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0000ed20 con 0x7efe0815d3b0 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0815a520 con 0x7efe0815d3b0 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efdfd7fa700 1 -- 172.21.15.7:0/2897652132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00014670 con 0x7efe0815d3b0 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.502+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0816a950 con 0x7efe0815d3b0 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.503+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0815aa40 con 0x7efe0815d3b0 2023-10-07T07:59:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.504+0000 7efdfd7fa700 1 -- 172.21.15.7:0/2897652132 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe000149e0 con 0x7efe0815d3b0 2023-10-07T07:59:44.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.504+0000 7efdfd7fa700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efdec066f30 0x7efdec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.504+0000 7efdfd7fa700 1 -- 172.21.15.7:0/2897652132 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7efe000848d0 con 0x7efe0815d3b0 2023-10-07T07:59:44.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.504+0000 7efe0d3d0700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efdec066f30 0x7efdec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.505+0000 7efe0d3d0700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efdec066f30 0x7efdec0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efe040093f0 tx=0x7efe04016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.508+0000 7efdfd7fa700 1 -- 172.21.15.7:0/2897652132 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe0815aa40 con 0x7efe0815d3b0 2023-10-07T07:59:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.710+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efe0815aa40 con 0x7efe0815d3b0 2023-10-07T07:59:44.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.714+0000 7efdfd7fa700 1 -- 172.21.15.7:0/2897652132 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7efe0815aa40 con 0x7efe0815d3b0 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efdec066f30 msgr2=0x7efdec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efdec066f30 0x7efdec0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efe040093f0 tx=0x7efe04016000 comp rx=0 tx=0).stop 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 msgr2=0x7efe080caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 0x7efe080caae0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efe00009fa0 tx=0x7efe00009900 comp rx=0 tx=0).stop 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 shutdown_connections 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7efdec066f30 0x7efdec0693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0815d3b0 0x7efe080caae0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe08069c90 0x7efe080af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 --2- 172.21.15.7:0/2897652132 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0806b610 0x7efe0816a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 >> 172.21.15.7:0/2897652132 conn(0x7efe08067ac0 msgr2=0x7efe080afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.716+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 shutdown_connections 2023-10-07T07:59:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.717+0000 7efe0f634700 1 -- 172.21.15.7:0/2897652132 wait complete. 2023-10-07T07:59:44.731 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-07T07:59:44.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T07:59:44.775 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T07:59:44.828 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-07T07:59:44.829 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-07T07:59:44.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:59:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.964+0000 7fbb7a316700 1 Processor -- start 2023-10-07T07:59:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.965+0000 7fbb7a316700 1 -- start start 2023-10-07T07:59:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.965+0000 7fbb7a316700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb7406bd30 0x7fbb740c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.965+0000 7fbb7a316700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb74068100 con 0x7fbb74159310 2023-10-07T07:59:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.965+0000 7fbb7a316700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb74068350 con 0x7fbb740ced30 2023-10-07T07:59:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.965+0000 7fbb7a316700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb74068630 con 0x7fbb7406bd30 2023-10-07T07:59:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb73fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbb74159310 0x7fbb740c3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54566/0 (socket says 172.21.15.7:54566) 2023-10-07T07:59:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb73fff700 1 -- 172.21.15.7:0/2212147643 learned_addr learned my addr 172.21.15.7:0/2212147643 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb788b3700 1 --2- 172.21.15.7:0/2212147643 >> v2:172.21.15.172:3300/0 conn(0x7fbb7406bd30 0x7fbb740c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011649308 0 0) 0x7fbb74068100 con 0x7fbb74159310 2023-10-07T07:59:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb5c003570 con 0x7fbb74159310 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005335689 0 0) 0x7fbb74068350 con 0x7fbb740ced30 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb74068100 con 0x7fbb740ced30 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3641325762 0 0) 0x7fbb5c003570 con 0x7fbb74159310 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 >> v2:172.21.15.172:3300/0 conn(0x7fbb7406bd30 msgr2=0x7fbb740c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/2212147643 >> v2:172.21.15.172:3300/0 conn(0x7fbb7406bd30 0x7fbb740c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 >> v1:172.21.15.172:6789/0 conn(0x7fbb740ced30 legacy=0x7fbb7406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74068910 con 0x7fbb74159310 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7fbb64001470 con 0x7fbb74159310 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb788b3700 1 --2- 172.21.15.7:0/2212147643 >> v2:172.21.15.172:3300/0 conn(0x7fbb7406bd30 0x7fbb740c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 >> v1:172.21.15.7:6789/0 conn(0x7fbb74159310 legacy=0x7fbb740c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c003bf0 0x7fbb5c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5c004810 0x7fbb5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb5c0051c0 0x7fbb5c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74068630 con 0x7fbb5c003bf0 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c00c0b0 con 0x7fbb5c004810 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.966+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c00c270 con 0x7fbb5c0051c0 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb73fff700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c003bf0 0x7fbb5c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb737fe700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5c004810 0x7fbb5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb788b3700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb5c0051c0 0x7fbb5c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb73fff700 1 -- 172.21.15.7:0/2212147643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb5c0051c0 msgr2=0x7fbb5c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb73fff700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb5c0051c0 0x7fbb5c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb73fff700 1 -- 172.21.15.7:0/2212147643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5c004810 msgr2=0x7fbb5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb73fff700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5c004810 0x7fbb5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb788b3700 1 -- 172.21.15.7:0/2212147643 reap_dead start 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb73fff700 1 -- 172.21.15.7:0/2212147643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c003570 con 0x7fbb5c003bf0 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb73fff700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c003bf0 0x7fbb5c003fe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c2d0 tx=0x7fbb64001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943cec0586656db0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400cbc0 con 0x7fbb5c003bf0 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6400cd20 con 0x7fbb5c003bf0 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.967+0000 7fbb72ffd700 1 -- 172.21.15.7:0/2212147643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64005b70 con 0x7fbb5c003bf0 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.968+0000 7fbb7a316700 1 -- 172.21.15.7:0/2212147643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c003bf0 msgr2=0x7fbb5c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.968+0000 7fbb7a316700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c003bf0 0x7fbb5c003fe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c2d0 tx=0x7fbb64001e10 comp rx=0 tx=0).stop 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.968+0000 7fbb7a316700 1 -- 172.21.15.7:0/2212147643 shutdown_connections 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.968+0000 7fbb7a316700 1 --2- 172.21.15.7:0/2212147643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c003bf0 0x7fbb5c003fe0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.968+0000 7fbb7a316700 1 -- 172.21.15.7:0/2212147643 >> 172.21.15.7:0/2212147643 conn(0x7fbb74067ac0 msgr2=0x7fbb740ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.968+0000 7fbb7a316700 1 -- 172.21.15.7:0/2212147643 shutdown_connections 2023-10-07T07:59:44.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.968+0000 7fbb7a316700 1 -- 172.21.15.7:0/2212147643 wait complete. 2023-10-07T07:59:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.969+0000 7fbb7a316700 1 Processor -- start 2023-10-07T07:59:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.970+0000 7fbb7a316700 1 -- start start 2023-10-07T07:59:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb7a316700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb5c003bf0 0x7fbb740cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb7a316700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb74154720 0x7fbb74154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb7a316700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415bdc0 0x7fbb7415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb7a316700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb741550d0 con 0x7fbb7415bdc0 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb7a316700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7416d8b0 con 0x7fbb74154720 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb7a316700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7416cca0 con 0x7fbb5c003bf0 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb788b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415bdc0 0x7fbb7415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb788b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415bdc0 0x7fbb7415c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47606/0 (socket says 172.21.15.7:47606) 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb788b3700 1 -- 172.21.15.7:0/389294099 learned_addr learned my addr 172.21.15.7:0/389294099 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb737fe700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb74154720 0x7fbb74154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb73fff700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb5c003bf0 0x7fbb740cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb788b3700 1 -- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb5c003bf0 msgr2=0x7fbb740cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb788b3700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb5c003bf0 0x7fbb740cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb788b3700 1 -- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb74154720 msgr2=0x7fbb74154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb788b3700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb74154720 0x7fbb74154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:44.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb788b3700 1 -- 172.21.15.7:0/389294099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7416cf80 con 0x7fbb7415bdc0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.971+0000 7fbb788b3700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415bdc0 0x7fbb7415c210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbb74063d10 tx=0x7fbb6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.972+0000 7fbb5bfff700 1 -- 172.21.15.7:0/389294099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00e070 con 0x7fbb7415bdc0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.972+0000 7fbb5bfff700 1 -- 172.21.15.7:0/389294099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c011d00 con 0x7fbb7415bdc0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.972+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7416d260 con 0x7fbb7415bdc0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.972+0000 7fbb5bfff700 1 -- 172.21.15.7:0/389294099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0166e0 con 0x7fbb7415bdc0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.972+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7415c750 con 0x7fbb7415bdc0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.973+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb740716c0 con 0x7fbb7415bdc0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.973+0000 7fbb5bfff700 1 -- 172.21.15.7:0/389294099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb6c016a90 con 0x7fbb7415bdc0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.974+0000 7fbb5bfff700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb60066f30 0x7fbb600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.974+0000 7fbb5bfff700 1 -- 172.21.15.7:0/389294099 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbb6c084e20 con 0x7fbb7415bdc0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.974+0000 7fbb73fff700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb60066f30 0x7fbb600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:44.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.974+0000 7fbb73fff700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb60066f30 0x7fbb600693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb64008fd0 tx=0x7fbb64015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:44.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:44.978+0000 7fbb5bfff700 1 -- 172.21.15.7:0/389294099 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb6c055ee0 con 0x7fbb7415bdc0 2023-10-07T07:59:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.229+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb740cea10 con 0x7fbb7415bdc0 2023-10-07T07:59:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.229+0000 7fbb5bfff700 1 -- 172.21.15.7:0/389294099 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3526 (secure 0 0 0) 0x7fbb6c059090 con 0x7fbb7415bdc0 2023-10-07T07:59:45.231 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:45.231 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/580346028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":580346028},{"type":"v1","addr":"172.21.15.172:6837","nonce":580346028}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3012863722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3012863722},{"type":"v1","addr":"172.21.15.7:6835","nonce":3012863722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4947,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3733075096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3733075096},{"type":"v1","addr":"172.21.15.172:6836","nonce":3733075096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2413703644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2413703644},{"type":"v1","addr":"172.21.15.7:6834","nonce":2413703644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-10-07T07:59:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb60066f30 msgr2=0x7fbb600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb60066f30 0x7fbb600693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb64008fd0 tx=0x7fbb64015000 comp rx=0 tx=0).stop 2023-10-07T07:59:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415bdc0 msgr2=0x7fbb7415c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415bdc0 0x7fbb7415c210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbb74063d10 tx=0x7fbb6c00b040 comp rx=0 tx=0).stop 2023-10-07T07:59:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 shutdown_connections 2023-10-07T07:59:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbb60066f30 0x7fbb600693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415bdc0 0x7fbb7415c210 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb5c003bf0 0x7fbb740cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 --2- 172.21.15.7:0/389294099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb74154720 0x7fbb74154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 >> 172.21.15.7:0/389294099 conn(0x7fbb74067ac0 msgr2=0x7fbb740c98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 shutdown_connections 2023-10-07T07:59:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.232+0000 7fbb7a316700 1 -- 172.21.15.7:0/389294099 wait complete. 2023-10-07T07:59:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2023-10-07T07:59:45.247 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-07T07:59:45.247 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-07T07:59:45.247 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-07T07:59:45.247 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T07:59:45.247 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-07T07:59:45.247 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-07T07:59:45.248 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-07T07:59:45.248 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T07:59:45.248 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-07T07:59:45.248 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-07T07:59:45.248 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-07T07:59:45.248 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T07:59:45.248 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-07T07:59:45.249 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-07T07:59:45.249 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-07T07:59:45.249 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T07:59:45.257 INFO:tasks.ceph.mds.a:Stopped 2023-10-07T07:59:45.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-07T07:59:45.258 INFO:tasks.ceph.mds.c:Stopped 2023-10-07T07:59:45.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-07T07:59:45.260 INFO:tasks.ceph.mds.b:Stopped 2023-10-07T07:59:45.260 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-07T07:59:45.261 INFO:tasks.ceph.mds.d:Stopped 2023-10-07T07:59:45.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-07T07:59:45.299 INFO:tasks.ceph.mds.a:Started 2023-10-07T07:59:45.300 DEBUG:teuthology.parallel:result is None 2023-10-07T07:59:45.300 INFO:tasks.ceph.mds.c:Started 2023-10-07T07:59:45.301 DEBUG:teuthology.parallel:result is None 2023-10-07T07:59:45.303 INFO:tasks.ceph.mds.b:Started 2023-10-07T07:59:45.303 DEBUG:teuthology.parallel:result is None 2023-10-07T07:59:45.303 INFO:tasks.ceph.mds.d:Started 2023-10-07T07:59:45.303 DEBUG:teuthology.parallel:result is None 2023-10-07T07:59:45.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-07T07:59:45.386 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2023-10-07T07:59:45.387 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2023-10-07T07:59:45.390 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2023-10-07T07:59:45.390 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2023-10-07T07:59:45.391 INFO:tasks.ceph.mds.d.smithi172.stderr:2023-10-07T07:59:45.390+0000 7fa44201a700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-07T07:59:45.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.443+0000 7ff639eb9700 1 Processor -- start 2023-10-07T07:59:45.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.443+0000 7ff639eb9700 1 -- start start 2023-10-07T07:59:45.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.444+0000 7ff639eb9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff634157470 0x7ff63406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:45.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.444+0000 7ff639eb9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6340c8f10 con 0x7ff6341570a0 2023-10-07T07:59:45.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.444+0000 7ff639eb9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6340c3b80 con 0x7ff63406bb60 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.444+0000 7ff639eb9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63406bfe0 con 0x7ff634157470 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.444+0000 7ff6337fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff6341570a0 0x7ff6340c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54576/0 (socket says 172.21.15.7:54576) 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.444+0000 7ff6337fe700 1 -- 172.21.15.7:0/2809975609 learned_addr learned my addr 172.21.15.7:0/2809975609 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.444+0000 7ff632ffd700 1 --2- 172.21.15.7:0/2809975609 >> v2:172.21.15.172:3300/0 conn(0x7ff634157470 0x7ff63406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.445+0000 7ff6327fc700 1 -- 172.21.15.7:0/2809975609 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973806396 0 0) 0x7ff624001110 con 0x7ff6341570a0 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.445+0000 7ff632ffd700 1 -- 172.21.15.7:0/2809975609 >> v1:172.21.15.172:6789/0 conn(0x7ff63406bb60 legacy=0x7ff6340c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.445+0000 7ff632ffd700 1 -- 172.21.15.7:0/2809975609 >> v1:172.21.15.7:6789/0 conn(0x7ff6341570a0 legacy=0x7ff6340c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.445+0000 7ff632ffd700 1 -- 172.21.15.7:0/2809975609 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340c2f70 con 0x7ff634157470 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.445+0000 7ff632ffd700 1 --2- 172.21.15.7:0/2809975609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff634157470 0x7ff63406b620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0098e0 tx=0x7ff61c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d0359fdba06324 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.445+0000 7ff6327fc700 1 -- 172.21.15.7:0/2809975609 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c009e80 con 0x7ff634157470 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.445+0000 7ff6327fc700 1 -- 172.21.15.7:0/2809975609 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c00b820 con 0x7ff634157470 2023-10-07T07:59:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.445+0000 7ff6327fc700 1 -- 172.21.15.7:0/2809975609 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c00bba0 con 0x7ff634157470 2023-10-07T07:59:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.446+0000 7ff639eb9700 1 -- 172.21.15.7:0/2809975609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff634157470 msgr2=0x7ff63406b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.446+0000 7ff639eb9700 1 --2- 172.21.15.7:0/2809975609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff634157470 0x7ff63406b620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0098e0 tx=0x7ff61c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.446+0000 7ff639eb9700 1 -- 172.21.15.7:0/2809975609 shutdown_connections 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.446+0000 7ff639eb9700 1 --2- 172.21.15.7:0/2809975609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff634157470 0x7ff63406b620 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.446+0000 7ff639eb9700 1 -- 172.21.15.7:0/2809975609 >> 172.21.15.7:0/2809975609 conn(0x7ff634067ac0 msgr2=0x7ff6340ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.446+0000 7ff639eb9700 1 -- 172.21.15.7:0/2809975609 shutdown_connections 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.446+0000 7ff639eb9700 1 -- 172.21.15.7:0/2809975609 wait complete. 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.447+0000 7ff639eb9700 1 Processor -- start 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.447+0000 7ff639eb9700 1 -- start start 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff639eb9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63406bb60 0x7ff6340b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff639eb9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6341570a0 0x7ff63416a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff639eb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff634157470 0x7ff6340b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff639eb9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63413a340 con 0x7ff634157470 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff639eb9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340b15f0 con 0x7ff63406bb60 2023-10-07T07:59:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff639eb9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340b1fc0 con 0x7ff6341570a0 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff633fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff634157470 0x7ff6340b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff633fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff634157470 0x7ff6340b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47630/0 (socket says 172.21.15.7:47630) 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff633fff700 1 -- 172.21.15.7:0/2286103462 learned_addr learned my addr 172.21.15.7:0/2286103462 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff6337fe700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63406bb60 0x7ff6340b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff632ffd700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6341570a0 0x7ff63416a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff6337fe700 1 -- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6341570a0 msgr2=0x7ff63416a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff6337fe700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6341570a0 0x7ff63416a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff6337fe700 1 -- 172.21.15.7:0/2286103462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff634157470 msgr2=0x7ff6340b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff6337fe700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff634157470 0x7ff6340b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff6337fe700 1 -- 172.21.15.7:0/2286103462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c009420 con 0x7ff63406bb60 2023-10-07T07:59:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff633fff700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff634157470 0x7ff6340b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:59:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.448+0000 7ff632ffd700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6341570a0 0x7ff63416a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:59:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.449+0000 7ff6337fe700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63406bb60 0x7ff6340b1a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c810 tx=0x7ff624007b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.449+0000 7ff62b7fe700 1 -- 172.21.15.7:0/2286103462 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624005b30 con 0x7ff63406bb60 2023-10-07T07:59:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.449+0000 7ff62b7fe700 1 -- 172.21.15.7:0/2286103462 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624005c90 con 0x7ff63406bb60 2023-10-07T07:59:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.449+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63416a980 con 0x7ff63406bb60 2023-10-07T07:59:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.449+0000 7ff62b7fe700 1 -- 172.21.15.7:0/2286103462 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240072e0 con 0x7ff63406bb60 2023-10-07T07:59:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.449+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff634069f10 con 0x7ff63406bb60 2023-10-07T07:59:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.450+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6340b18d0 con 0x7ff63406bb60 2023-10-07T07:59:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.450+0000 7ff62b7fe700 1 -- 172.21.15.7:0/2286103462 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6240076d0 con 0x7ff63406bb60 2023-10-07T07:59:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.451+0000 7ff62b7fe700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff614066f30 0x7ff6140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.451+0000 7ff62b7fe700 1 -- 172.21.15.7:0/2286103462 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff6240832f0 con 0x7ff63406bb60 2023-10-07T07:59:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.451+0000 7ff632ffd700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff614066f30 0x7ff6140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.452+0000 7ff632ffd700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff614066f30 0x7ff6140693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0030c0 tx=0x7ff61c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.456+0000 7ff62b7fe700 1 -- 172.21.15.7:0/2286103462 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff62404b5a0 con 0x7ff63406bb60 2023-10-07T07:59:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:45.677+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff6340c8f10 con 0x7ff63406bb60 2023-10-07T07:59:46.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.351+0000 7ff62b7fe700 1 -- 172.21.15.7:0/2286103462 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff6340c8f10 con 0x7ff63406bb60 2023-10-07T07:59:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.354+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff614066f30 msgr2=0x7ff6140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.354+0000 7ff639eb9700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff614066f30 0x7ff6140693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0030c0 tx=0x7ff61c016000 comp rx=0 tx=0).stop 2023-10-07T07:59:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.354+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63406bb60 msgr2=0x7ff6340b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.354+0000 7ff639eb9700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63406bb60 0x7ff6340b1a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c810 tx=0x7ff624007b90 comp rx=0 tx=0).stop 2023-10-07T07:59:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.355+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 shutdown_connections 2023-10-07T07:59:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.355+0000 7ff639eb9700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff614066f30 0x7ff6140693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.355+0000 7ff639eb9700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff634157470 0x7ff6340b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.355+0000 7ff639eb9700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63406bb60 0x7ff6340b1a80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.355+0000 7ff639eb9700 1 --2- 172.21.15.7:0/2286103462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6341570a0 0x7ff63416a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.355+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 >> 172.21.15.7:0/2286103462 conn(0x7ff634067ac0 msgr2=0x7ff6341591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.355+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 shutdown_connections 2023-10-07T07:59:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.355+0000 7ff639eb9700 1 -- 172.21.15.7:0/2286103462 wait complete. 2023-10-07T07:59:46.357 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2023-10-07T07:59:46.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-07T07:59:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.508+0000 7f0b87995700 1 Processor -- start 2023-10-07T07:59:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.508+0000 7f0b87995700 1 -- start start 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b87995700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b801570a0 0x7f0b800c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b87995700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b80157470 0x7f0b8006b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b87995700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b800c3b80 con 0x7f0b8006bb60 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b87995700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8006bfe0 con 0x7f0b80157470 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b87995700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800c2f70 con 0x7f0b801570a0 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b84f30700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b80157470 0x7f0b8006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b85f32700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0b8006bb60 0x7f0b8006cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54586/0 (socket says 172.21.15.7:54586) 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b85f32700 1 -- 172.21.15.7:0/1383629705 learned_addr learned my addr 172.21.15.7:0/1383629705 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b84f30700 1 -- 172.21.15.7:0/1383629705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b801570a0 msgr2=0x7f0b800c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b85731700 1 --2- 172.21.15.7:0/1383629705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b801570a0 0x7f0b800c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.509+0000 7f0b77fff700 1 -- 172.21.15.7:0/1383629705 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402507125 0 0) 0x7f0b800c3b80 con 0x7f0b8006bb60 2023-10-07T07:59:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b84f30700 1 --2- 172.21.15.7:0/1383629705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b801570a0 0x7f0b800c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b84f30700 1 -- 172.21.15.7:0/1383629705 >> v1:172.21.15.7:6789/0 conn(0x7f0b8006bb60 legacy=0x7f0b8006cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b84f30700 1 -- 172.21.15.7:0/1383629705 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b800c3250 con 0x7f0b80157470 2023-10-07T07:59:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b85731700 1 --2- 172.21.15.7:0/1383629705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b801570a0 0x7f0b800c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b84f30700 1 --2- 172.21.15.7:0/1383629705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b80157470 0x7f0b8006b620 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0b700098e0 tx=0x7f0b700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e60f344c4b5f97 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b77fff700 1 -- 172.21.15.7:0/1383629705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70009e80 con 0x7f0b80157470 2023-10-07T07:59:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b77fff700 1 -- 172.21.15.7:0/1383629705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7000b860 con 0x7f0b80157470 2023-10-07T07:59:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b77fff700 1 -- 172.21.15.7:0/1383629705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000bbe0 con 0x7f0b80157470 2023-10-07T07:59:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b87995700 1 -- 172.21.15.7:0/1383629705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b80157470 msgr2=0x7f0b8006b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.510+0000 7f0b87995700 1 --2- 172.21.15.7:0/1383629705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b80157470 0x7f0b8006b620 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0b700098e0 tx=0x7f0b700091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.511+0000 7f0b87995700 1 -- 172.21.15.7:0/1383629705 shutdown_connections 2023-10-07T07:59:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.511+0000 7f0b87995700 1 --2- 172.21.15.7:0/1383629705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b801570a0 0x7f0b800c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.511+0000 7f0b87995700 1 --2- 172.21.15.7:0/1383629705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b80157470 0x7f0b8006b620 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.511+0000 7f0b87995700 1 -- 172.21.15.7:0/1383629705 >> 172.21.15.7:0/1383629705 conn(0x7f0b80067ac0 msgr2=0x7f0b800ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.511+0000 7f0b87995700 1 -- 172.21.15.7:0/1383629705 shutdown_connections 2023-10-07T07:59:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.511+0000 7f0b87995700 1 -- 172.21.15.7:0/1383629705 wait complete. 2023-10-07T07:59:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.512+0000 7f0b87995700 1 Processor -- start 2023-10-07T07:59:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.512+0000 7f0b87995700 1 -- start start 2023-10-07T07:59:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b87995700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b8006bb60 0x7f0b800c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b87995700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b801570a0 0x7f0b800c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b87995700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b80157470 0x7f0b800c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b87995700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800cca50 con 0x7f0b801570a0 2023-10-07T07:59:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b87995700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800ccd30 con 0x7f0b80157470 2023-10-07T07:59:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b87995700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800cd010 con 0x7f0b8006bb60 2023-10-07T07:59:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b84f30700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b801570a0 0x7f0b800c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b84f30700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b801570a0 0x7f0b800c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47640/0 (socket says 172.21.15.7:47640) 2023-10-07T07:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b84f30700 1 -- 172.21.15.7:0/3674481841 learned_addr learned my addr 172.21.15.7:0/3674481841 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b85f32700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b80157470 0x7f0b800c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b85731700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b8006bb60 0x7f0b800c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b84f30700 1 -- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b8006bb60 msgr2=0x7f0b800c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b84f30700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b8006bb60 0x7f0b800c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b84f30700 1 -- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b80157470 msgr2=0x7f0b800c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b84f30700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b80157470 0x7f0b800c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.513+0000 7f0b84f30700 1 -- 172.21.15.7:0/3674481841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70009420 con 0x7f0b801570a0 2023-10-07T07:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.514+0000 7f0b84f30700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b801570a0 0x7f0b800c5cd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0b800cbc20 tx=0x7f0b70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.514+0000 7f0b757fa700 1 -- 172.21.15.7:0/3674481841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70000cc0 con 0x7f0b801570a0 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.514+0000 7f0b757fa700 1 -- 172.21.15.7:0/3674481841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7000b9d0 con 0x7f0b801570a0 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.514+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b800cd2f0 con 0x7f0b801570a0 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.514+0000 7f0b757fa700 1 -- 172.21.15.7:0/3674481841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70025670 con 0x7f0b801570a0 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.514+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b801560c0 con 0x7f0b801570a0 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.515+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b800716c0 con 0x7f0b801570a0 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.516+0000 7f0b757fa700 1 -- 172.21.15.7:0/3674481841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b70023070 con 0x7f0b801570a0 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.516+0000 7f0b757fa700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0b64066f30 0x7f0b640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.516+0000 7f0b757fa700 1 -- 172.21.15.7:0/3674481841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0b700856e0 con 0x7f0b801570a0 2023-10-07T07:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.516+0000 7f0b85731700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0b64066f30 0x7f0b640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.517+0000 7f0b85731700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0b64066f30 0x7f0b640693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00adc0 tx=0x7f0b7c006f90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.521+0000 7f0b757fa700 1 -- 172.21.15.7:0/3674481841 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b70052ca0 con 0x7f0b801570a0 2023-10-07T07:59:46.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.809+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0b800cea10 con 0x7f0b801570a0 2023-10-07T07:59:46.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.809+0000 7f0b757fa700 1 -- 172.21.15.7:0/3674481841 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0b700562c0 con 0x7f0b801570a0 2023-10-07T07:59:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:46.811 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:59:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2023-10-07T07:59:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2023-10-07T07:59:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlCy6iHBAA0pyZcla7yG20KJxMwhhFVg==", 2023-10-07T07:59:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:59:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:59:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:59:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2023-10-07T07:59:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlUxYOKhAAuXUwfiu6GTlHPau/HLhgJg==", 2023-10-07T07:59:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:59:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:59:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:59:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2023-10-07T07:59:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFluHPbJBAAdTCLab+QEn7jhGF253MA4A==", 2023-10-07T07:59:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:59:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:59:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:59:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2023-10-07T07:59:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlNaU3MRAAqpmLriqCxUsXuAyL5Cmbxw==", 2023-10-07T07:59:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:59:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:59:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:59:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2023-10-07T07:59:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBODyFlkgzdGRAAUXBLhY69mpInyEg/EZ1bbg==", 2023-10-07T07:59:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:59:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:59:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2023-10-07T07:59:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBRDyFl95iAGBAA8mk47fzfwi2BDWqxoJIAGA==", 2023-10-07T07:59:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:59:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:59:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2023-10-07T07:59:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBUDyFlpgWXFxAABhoYDIRsLPfY19Z76zlRfg==", 2023-10-07T07:59:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:59:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:59:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2023-10-07T07:59:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBXDyFl9VThFhAAKbJJRsjjubjWK5xpt7j6Vg==", 2023-10-07T07:59:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:59:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:59:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2023-10-07T07:59:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBdDyFlcKXTGhAAPW0NNpqt2aoxJZUSdBtJEw==", 2023-10-07T07:59:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:59:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:59:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2023-10-07T07:59:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBgDyFlYdmOGRAAO4UiW/o3zkFHKLo0A/Fl0g==", 2023-10-07T07:59:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:59:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:59:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2023-10-07T07:59:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBjDyFlxgq2FxAAar/DNRzL+QMVm0sZJ6dY1A==", 2023-10-07T07:59:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:59:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:59:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2023-10-07T07:59:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBmDyFlZyNcFhAAfEAyrTMB6AkewTly3i3vVQ==", 2023-10-07T07:59:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2023-10-07T07:59:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2023-10-07T07:59:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2023-10-07T07:59:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlom77NRAA6mby9LFc/5XVOSXctSuffA==", 2023-10-07T07:59:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.829 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:59:46.829 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2023-10-07T07:59:46.829 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-07T07:59:46.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.829 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2023-10-07T07:59:46.830 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBNDyFlGLtzARAAh3o32z7sOz1RBqXINgae6g==", 2023-10-07T07:59:46.830 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.830 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:59:46.830 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-07T07:59:46.830 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-07T07:59:46.830 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-07T07:59:46.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.831 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2023-10-07T07:59:46.831 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlrCK0ORAAWpaeoXzMPo3ko8Yrat1bEg==", 2023-10-07T07:59:46.831 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.831 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:59:46.831 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-07T07:59:46.831 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-07T07:59:46.831 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-07T07:59:46.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.832 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2023-10-07T07:59:46.832 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBNDyFl/YjxBhAAivYmiSK5+fdwyFmyMeWI+A==", 2023-10-07T07:59:46.832 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.832 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2023-10-07T07:59:46.833 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2023-10-07T07:59:46.833 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2023-10-07T07:59:46.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2023-10-07T07:59:46.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.833 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2023-10-07T07:59:46.833 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBLDyFl3DSBOBAARv3edVYI1sfQLkk7iSk9IQ==", 2023-10-07T07:59:46.834 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.834 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2023-10-07T07:59:46.834 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2023-10-07T07:59:46.834 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2023-10-07T07:59:46.834 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.835 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2023-10-07T07:59:46.835 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlgeSpGBAAlq+bBWoPBk7KPt5+xL1z4w==", 2023-10-07T07:59:46.835 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.835 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2023-10-07T07:59:46.835 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2023-10-07T07:59:46.836 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:46.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:46.836 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2023-10-07T07:59:46.836 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQBMDyFlhkIGFRAAWzKzlNAmIb3PXAlOaZMtYA==", 2023-10-07T07:59:46.836 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2023-10-07T07:59:46.837 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2023-10-07T07:59:46.837 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2023-10-07T07:59:46.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2023-10-07T07:59:46.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:46.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2023-10-07T07:59:46.838 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:59:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0b64066f30 msgr2=0x7f0b640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0b64066f30 0x7f0b640693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00adc0 tx=0x7f0b7c006f90 comp rx=0 tx=0).stop 2023-10-07T07:59:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b801570a0 msgr2=0x7f0b800c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b801570a0 0x7f0b800c5cd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0b800cbc20 tx=0x7f0b70017000 comp rx=0 tx=0).stop 2023-10-07T07:59:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 shutdown_connections 2023-10-07T07:59:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0b64066f30 0x7f0b640693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b80157470 0x7f0b800c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b8006bb60 0x7f0b800c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 --2- 172.21.15.7:0/3674481841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b801570a0 0x7f0b800c5cd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 >> 172.21.15.7:0/3674481841 conn(0x7f0b80067ac0 msgr2=0x7f0b80139930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:46.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 shutdown_connections 2023-10-07T07:59:46.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.812+0000 7f0b87995700 1 -- 172.21.15.7:0/3674481841 wait complete. 2023-10-07T07:59:46.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:59:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.974+0000 7f4112686700 1 Processor -- start 2023-10-07T07:59:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.974+0000 7f4112686700 1 -- start start 2023-10-07T07:59:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.975+0000 7f4112686700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f410c157470 0x7f410c06b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.975+0000 7f4112686700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410c0c8f10 con 0x7f410c06bb60 2023-10-07T07:59:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.975+0000 7f4112686700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410c0c3b80 con 0x7f410c1570a0 2023-10-07T07:59:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.975+0000 7f4112686700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f410c06bfe0 con 0x7f410c157470 2023-10-07T07:59:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.975+0000 7f410b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f410c157470 0x7f410c06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.975+0000 7f4110c23700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f410c06bb60 0x7f410c0c4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54590/0 (socket says 172.21.15.7:54590) 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.975+0000 7f4110c23700 1 -- 172.21.15.7:0/477633183 learned_addr learned my addr 172.21.15.7:0/477633183 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.976+0000 7f410b7fe700 1 -- 172.21.15.7:0/477633183 >> v1:172.21.15.172:6789/0 conn(0x7f410c1570a0 legacy=0x7f410c0c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.976+0000 7f410b7fe700 1 -- 172.21.15.7:0/477633183 >> v1:172.21.15.7:6789/0 conn(0x7f410c06bb60 legacy=0x7f410c0c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.976+0000 7f410b7fe700 1 -- 172.21.15.7:0/477633183 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0c2f70 con 0x7f410c157470 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.976+0000 7f410affd700 1 -- 172.21.15.7:0/477633183 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362299560 0 0) 0x7f4104001490 con 0x7f410c06bb60 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.976+0000 7f410b7fe700 1 --2- 172.21.15.7:0/477633183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c157470 0x7f410c06b620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41000098e0 tx=0x7f41000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b553a22d362cf0d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.976+0000 7f410affd700 1 -- 172.21.15.7:0/477633183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100009e80 con 0x7f410c157470 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.976+0000 7f410affd700 1 -- 172.21.15.7:0/477633183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410000b860 con 0x7f410c157470 2023-10-07T07:59:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.976+0000 7f410affd700 1 -- 172.21.15.7:0/477633183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410000bbe0 con 0x7f410c157470 2023-10-07T07:59:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.977+0000 7f4112686700 1 -- 172.21.15.7:0/477633183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c157470 msgr2=0x7f410c06b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.977+0000 7f4112686700 1 --2- 172.21.15.7:0/477633183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c157470 0x7f410c06b620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41000098e0 tx=0x7f41000091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.977+0000 7f4112686700 1 -- 172.21.15.7:0/477633183 shutdown_connections 2023-10-07T07:59:46.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.977+0000 7f4112686700 1 --2- 172.21.15.7:0/477633183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c157470 0x7f410c06b620 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.977+0000 7f4112686700 1 -- 172.21.15.7:0/477633183 >> 172.21.15.7:0/477633183 conn(0x7f410c067ac0 msgr2=0x7f410c0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:46.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.977+0000 7f4112686700 1 -- 172.21.15.7:0/477633183 shutdown_connections 2023-10-07T07:59:46.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.977+0000 7f4112686700 1 -- 172.21.15.7:0/477633183 wait complete. 2023-10-07T07:59:46.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.978+0000 7f4112686700 1 Processor -- start 2023-10-07T07:59:46.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.978+0000 7f4112686700 1 -- start start 2023-10-07T07:59:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f4112686700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f410c06bb60 0x7f410c0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f4112686700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c1570a0 0x7f410c0c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f4112686700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f410c157470 0x7f410c0c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f4112686700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c13e4a0 con 0x7f410c1570a0 2023-10-07T07:59:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f4112686700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0c7650 con 0x7f410c06bb60 2023-10-07T07:59:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f4112686700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0b3b10 con 0x7f410c157470 2023-10-07T07:59:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f410b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c1570a0 0x7f410c0c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f410b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c1570a0 0x7f410c0c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47652/0 (socket says 172.21.15.7:47652) 2023-10-07T07:59:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f410b7fe700 1 -- 172.21.15.7:0/3383792235 learned_addr learned my addr 172.21.15.7:0/3383792235 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f4110c23700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f410c157470 0x7f410c0c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.979+0000 7f410bfff700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f410c06bb60 0x7f410c0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f410b7fe700 1 -- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f410c157470 msgr2=0x7f410c0c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f410b7fe700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f410c157470 0x7f410c0c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f410b7fe700 1 -- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f410c06bb60 msgr2=0x7f410c0c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f410b7fe700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f410c06bb60 0x7f410c0c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f410b7fe700 1 -- 172.21.15.7:0/3383792235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100009420 con 0x7f410c1570a0 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f410b7fe700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c1570a0 0x7f410c0c6bd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f410000b3c0 tx=0x7f4100017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f40f3fff700 1 -- 172.21.15.7:0/3383792235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410001f070 con 0x7f410c1570a0 2023-10-07T07:59:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f40f3fff700 1 -- 172.21.15.7:0/3383792235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410000b9d0 con 0x7f410c1570a0 2023-10-07T07:59:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c13db70 con 0x7f410c1570a0 2023-10-07T07:59:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f40f3fff700 1 -- 172.21.15.7:0/3383792235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100011670 con 0x7f410c1570a0 2023-10-07T07:59:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.980+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c13e130 con 0x7f410c1570a0 2023-10-07T07:59:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.981+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c0716c0 con 0x7f410c1570a0 2023-10-07T07:59:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.982+0000 7f40f3fff700 1 -- 172.21.15.7:0/3383792235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4100011a20 con 0x7f410c1570a0 2023-10-07T07:59:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.982+0000 7f40f3fff700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f40f8066f30 0x7f40f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.982+0000 7f40f3fff700 1 -- 172.21.15.7:0/3383792235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4100085540 con 0x7f410c1570a0 2023-10-07T07:59:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.982+0000 7f410bfff700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f40f8066f30 0x7f40f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.983+0000 7f410bfff700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f40f8066f30 0x7f40f80693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f40fc003a80 tx=0x7f40fc007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:46.987+0000 7f40f3fff700 1 -- 172.21.15.7:0/3383792235 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4100052b00 con 0x7f410c1570a0 2023-10-07T07:59:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.190+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f410c13e5e0 con 0x7f410c1570a0 2023-10-07T07:59:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.191+0000 7f40f3fff700 1 -- 172.21.15.7:0/3383792235 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f4100056120 con 0x7f410c1570a0 2023-10-07T07:59:47.192 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:47.193 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:59:46.355510+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:59:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.193+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f40f8066f30 msgr2=0x7f40f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.193+0000 7f4112686700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f40f8066f30 0x7f40f80693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f40fc003a80 tx=0x7f40fc007ec0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.193+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c1570a0 msgr2=0x7f410c0c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.193+0000 7f4112686700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c1570a0 0x7f410c0c6bd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f410000b3c0 tx=0x7f4100017000 comp rx=0 tx=0).stop 2023-10-07T07:59:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.193+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 shutdown_connections 2023-10-07T07:59:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.193+0000 7f4112686700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f40f8066f30 0x7f40f80693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.193+0000 7f4112686700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f410c157470 0x7f410c0c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.194+0000 7f4112686700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f410c06bb60 0x7f410c0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.194+0000 7f4112686700 1 --2- 172.21.15.7:0/3383792235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f410c1570a0 0x7f410c0c6bd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.194+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 >> 172.21.15.7:0/3383792235 conn(0x7f410c067ac0 msgr2=0x7f410c13c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:47.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.194+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 shutdown_connections 2023-10-07T07:59:47.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.194+0000 7f4112686700 1 -- 172.21.15.7:0/3383792235 wait complete. 2023-10-07T07:59:47.208 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T07:59:47.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T07:59:47.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.344+0000 7fce95522700 1 Processor -- start 2023-10-07T07:59:47.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.344+0000 7fce95522700 1 -- start start 2023-10-07T07:59:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.345+0000 7fce95522700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fce9015dbf0 0x7fce900afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.345+0000 7fce95522700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce900af810 0x7fce900ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.345+0000 7fce95522700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce900cf1e0 con 0x7fce900af440 2023-10-07T07:59:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.345+0000 7fce95522700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce900ce5d0 con 0x7fce900af810 2023-10-07T07:59:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.345+0000 7fce95522700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce900ce8b0 con 0x7fce9015dbf0 2023-10-07T07:59:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.346+0000 7fce8effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce900af440 0x7fce900cb2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54600/0 (socket says 172.21.15.7:54600) 2023-10-07T07:59:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.346+0000 7fce8f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce900af810 0x7fce900ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.346+0000 7fce8effd700 1 -- 172.21.15.7:0/2321938611 learned_addr learned my addr 172.21.15.7:0/2321938611 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.346+0000 7fce8e7fc700 1 --2- 172.21.15.7:0/2321938611 >> v2:172.21.15.172:3300/0 conn(0x7fce9015dbf0 0x7fce900afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.346+0000 7fce8f7fe700 1 -- 172.21.15.7:0/2321938611 >> v2:172.21.15.172:3300/0 conn(0x7fce9015dbf0 msgr2=0x7fce900afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.346+0000 7fce8f7fe700 1 --2- 172.21.15.7:0/2321938611 >> v2:172.21.15.172:3300/0 conn(0x7fce9015dbf0 0x7fce900afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.346+0000 7fce8f7fe700 1 -- 172.21.15.7:0/2321938611 >> v1:172.21.15.7:6789/0 conn(0x7fce900af440 legacy=0x7fce900cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.346+0000 7fce8f7fe700 1 -- 172.21.15.7:0/2321938611 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce900ceb90 con 0x7fce900af810 2023-10-07T07:59:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.346+0000 7fce8f7fe700 1 --2- 172.21.15.7:0/2321938611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce900af810 0x7fce900ccc70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fce90069fe0 tx=0x7fce88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b009d4a7ed575a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:47.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce8dffb700 1 -- 172.21.15.7:0/2321938611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800e070 con 0x7fce900af810 2023-10-07T07:59:47.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce8dffb700 1 -- 172.21.15.7:0/2321938611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8800bcc0 con 0x7fce900af810 2023-10-07T07:59:47.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce8dffb700 1 -- 172.21.15.7:0/2321938611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88012680 con 0x7fce900af810 2023-10-07T07:59:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce95522700 1 -- 172.21.15.7:0/2321938611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce900af810 msgr2=0x7fce900ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce95522700 1 --2- 172.21.15.7:0/2321938611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce900af810 0x7fce900ccc70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fce90069fe0 tx=0x7fce88009780 comp rx=0 tx=0).stop 2023-10-07T07:59:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce95522700 1 -- 172.21.15.7:0/2321938611 shutdown_connections 2023-10-07T07:59:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce95522700 1 --2- 172.21.15.7:0/2321938611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce900af810 0x7fce900ccc70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce95522700 1 --2- 172.21.15.7:0/2321938611 >> v2:172.21.15.172:3300/0 conn(0x7fce9015dbf0 0x7fce900afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce95522700 1 -- 172.21.15.7:0/2321938611 >> 172.21.15.7:0/2321938611 conn(0x7fce90069220 msgr2=0x7fce90069620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.349+0000 7fce95522700 1 -- 172.21.15.7:0/2321938611 shutdown_connections 2023-10-07T07:59:47.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.350+0000 7fce95522700 1 -- 172.21.15.7:0/2321938611 wait complete. 2023-10-07T07:59:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.351+0000 7fce95522700 1 Processor -- start 2023-10-07T07:59:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.351+0000 7fce95522700 1 -- start start 2023-10-07T07:59:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce95522700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce900af440 0x7fce90146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce95522700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce900af810 0x7fce9016a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce95522700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9015dbf0 0x7fce90158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce95522700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9015c8f0 con 0x7fce9015dbf0 2023-10-07T07:59:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce95522700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9015bce0 con 0x7fce900af810 2023-10-07T07:59:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce95522700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9015bfc0 con 0x7fce900af440 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce8f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9015dbf0 0x7fce90158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce8effd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce900af440 0x7fce90146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce8e7fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce900af810 0x7fce9016a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce8f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9015dbf0 0x7fce90158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47672/0 (socket says 172.21.15.7:47672) 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce8f7fe700 1 -- 172.21.15.7:0/4182574651 learned_addr learned my addr 172.21.15.7:0/4182574651 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.352+0000 7fce8f7fe700 1 -- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce900af440 msgr2=0x7fce90146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce8f7fe700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce900af440 0x7fce90146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce8f7fe700 1 -- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce900af810 msgr2=0x7fce9016a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce8f7fe700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce900af810 0x7fce9016a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce8f7fe700 1 -- 172.21.15.7:0/4182574651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce88009a20 con 0x7fce9015dbf0 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce8effd700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce900af440 0x7fce90146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:59:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce8f7fe700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9015dbf0 0x7fce90158920 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fce900678f0 tx=0x7fce88019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce76ffd700 1 -- 172.21.15.7:0/4182574651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8801d630 con 0x7fce9015dbf0 2023-10-07T07:59:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce76ffd700 1 -- 172.21.15.7:0/4182574651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8801d7d0 con 0x7fce9015dbf0 2023-10-07T07:59:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce76ffd700 1 -- 172.21.15.7:0/4182574651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8801db90 con 0x7fce9015dbf0 2023-10-07T07:59:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9015c2a0 con 0x7fce9015dbf0 2023-10-07T07:59:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.353+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce90147520 con 0x7fce9015dbf0 2023-10-07T07:59:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.354+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce9015caa0 con 0x7fce9015dbf0 2023-10-07T07:59:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.354+0000 7fce76ffd700 1 -- 172.21.15.7:0/4182574651 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce88024070 con 0x7fce9015dbf0 2023-10-07T07:59:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.355+0000 7fce76ffd700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fce7c066f30 0x7fce7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.355+0000 7fce76ffd700 1 -- 172.21.15.7:0/4182574651 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fce88086ad0 con 0x7fce9015dbf0 2023-10-07T07:59:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.355+0000 7fce8effd700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fce7c066f30 0x7fce7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.356+0000 7fce8effd700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fce7c066f30 0x7fce7c0693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce80007110 tx=0x7fce80007f10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.359+0000 7fce76ffd700 1 -- 172.21.15.7:0/4182574651 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce9015caa0 con 0x7fce9015dbf0 2023-10-07T07:59:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.557+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce9004c3c0 con 0x7fce9015dbf0 2023-10-07T07:59:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.558+0000 7fce76ffd700 1 -- 172.21.15.7:0/4182574651 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce88054090 con 0x7fce9015dbf0 2023-10-07T07:59:47.559 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T07:59:47.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fce7c066f30 msgr2=0x7fce7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fce7c066f30 0x7fce7c0693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce80007110 tx=0x7fce80007f10 comp rx=0 tx=0).stop 2023-10-07T07:59:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9015dbf0 msgr2=0x7fce90158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9015dbf0 0x7fce90158920 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fce900678f0 tx=0x7fce88019000 comp rx=0 tx=0).stop 2023-10-07T07:59:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 shutdown_connections 2023-10-07T07:59:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fce7c066f30 0x7fce7c0693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9015dbf0 0x7fce90158920 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce900af440 0x7fce90146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 --2- 172.21.15.7:0/4182574651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce900af810 0x7fce9016a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 >> 172.21.15.7:0/4182574651 conn(0x7fce90069220 msgr2=0x7fce901567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 shutdown_connections 2023-10-07T07:59:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.560+0000 7fce95522700 1 -- 172.21.15.7:0/4182574651 wait complete. 2023-10-07T07:59:47.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-07T07:59:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.706+0000 7ff5a47c3700 1 Processor -- start 2023-10-07T07:59:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.706+0000 7ff5a47c3700 1 -- start start 2023-10-07T07:59:47.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.707+0000 7ff5a47c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff59c159380 0x7ff59c0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.707+0000 7ff5a47c3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff59c0d0250 0x7ff59c0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.707+0000 7ff5a47c3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0b3b60 0x7ff59c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.707+0000 7ff5a47c3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59c0d0be0 con 0x7ff59c159380 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.707+0000 7ff5a47c3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59c0b4820 con 0x7ff59c0d0250 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.707+0000 7ff5a47c3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c0b5000 con 0x7ff59c0b3b60 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a255f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff59c159380 0x7ff59c0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a255f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff59c159380 0x7ff59c0b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47676/0 (socket says 172.21.15.7:47676) 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a255f700 1 -- 172.21.15.7:0/2106947658 learned_addr learned my addr 172.21.15.7:0/2106947658 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a2d60700 1 --2- 172.21.15.7:0/2106947658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0b3b60 0x7ff59c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a1d5e700 1 --2- 172.21.15.7:0/2106947658 >> v2:172.21.15.172:3300/0 conn(0x7ff59c0d0250 0x7ff59c0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a255f700 1 -- 172.21.15.7:0/2106947658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0b3b60 msgr2=0x7ff59c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a255f700 1 --2- 172.21.15.7:0/2106947658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0b3b60 0x7ff59c0b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a255f700 1 -- 172.21.15.7:0/2106947658 >> v2:172.21.15.172:3300/0 conn(0x7ff59c0d0250 msgr2=0x7ff59c0d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a255f700 1 --2- 172.21.15.7:0/2106947658 >> v2:172.21.15.172:3300/0 conn(0x7ff59c0d0250 0x7ff59c0d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a255f700 1 -- 172.21.15.7:0/2106947658 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c0b52e0 con 0x7ff59c159380 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a255f700 1 --2- 172.21.15.7:0/2106947658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c159380 0x7ff59c0b5ae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff59c067900 tx=0x7ff598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b4feade2a86922 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a155d700 1 -- 172.21.15.7:0/2106947658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59800b640 con 0x7ff59c159380 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a155d700 1 -- 172.21.15.7:0/2106947658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59800b7a0 con 0x7ff59c159380 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.708+0000 7ff5a155d700 1 -- 172.21.15.7:0/2106947658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59800bb60 con 0x7ff59c159380 2023-10-07T07:59:47.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.709+0000 7ff5a47c3700 1 -- 172.21.15.7:0/2106947658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c159380 msgr2=0x7ff59c0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.709+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/2106947658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c159380 0x7ff59c0b5ae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff59c067900 tx=0x7ff598009250 comp rx=0 tx=0).stop 2023-10-07T07:59:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.709+0000 7ff5a47c3700 1 -- 172.21.15.7:0/2106947658 shutdown_connections 2023-10-07T07:59:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.709+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/2106947658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0b3b60 0x7ff59c0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.709+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/2106947658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c159380 0x7ff59c0b5ae0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.709+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/2106947658 >> v2:172.21.15.172:3300/0 conn(0x7ff59c0d0250 0x7ff59c0d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.709+0000 7ff5a47c3700 1 -- 172.21.15.7:0/2106947658 >> 172.21.15.7:0/2106947658 conn(0x7ff59c067be0 msgr2=0x7ff59c067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.709+0000 7ff5a47c3700 1 -- 172.21.15.7:0/2106947658 shutdown_connections 2023-10-07T07:59:47.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.709+0000 7ff5a47c3700 1 -- 172.21.15.7:0/2106947658 wait complete. 2023-10-07T07:59:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.710+0000 7ff5a47c3700 1 Processor -- start 2023-10-07T07:59:47.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.710+0000 7ff5a47c3700 1 -- start start 2023-10-07T07:59:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.711+0000 7ff5a47c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0b3b60 0x7ff59c1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.711+0000 7ff5a47c3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0d0250 0x7ff59c14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.711+0000 7ff5a47c3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff59c159380 0x7ff59c14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.711+0000 7ff5a47c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c14fab0 con 0x7ff59c0b3b60 2023-10-07T07:59:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.711+0000 7ff5a47c3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c156ad0 con 0x7ff59c159380 2023-10-07T07:59:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.711+0000 7ff5a47c3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c14e410 con 0x7ff59c0d0250 2023-10-07T07:59:47.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.711+0000 7ff5a255f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0b3b60 0x7ff59c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.711+0000 7ff5a255f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0b3b60 0x7ff59c1502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47682/0 (socket says 172.21.15.7:47682) 2023-10-07T07:59:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.711+0000 7ff5a255f700 1 -- 172.21.15.7:0/3660052398 learned_addr learned my addr 172.21.15.7:0/3660052398 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a1d5e700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0d0250 0x7ff59c14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a2d60700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff59c159380 0x7ff59c14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a255f700 1 -- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0d0250 msgr2=0x7ff59c14c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a255f700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0d0250 0x7ff59c14c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a255f700 1 -- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff59c159380 msgr2=0x7ff59c14cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a255f700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff59c159380 0x7ff59c14cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a255f700 1 -- 172.21.15.7:0/3660052398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980094f0 con 0x7ff59c0b3b60 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a2d60700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff59c159380 0x7ff59c14cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a255f700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0b3b60 0x7ff59c1502b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff59c06f850 tx=0x7ff598017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5927fc700 1 -- 172.21.15.7:0/3660052398 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598000d10 con 0x7ff59c0b3b60 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5927fc700 1 -- 172.21.15.7:0/3660052398 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59800bb20 con 0x7ff59c0b3b60 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c14e6f0 con 0x7ff59c0b3b60 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5927fc700 1 -- 172.21.15.7:0/3660052398 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598011670 con 0x7ff59c0b3b60 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.712+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c1507f0 con 0x7ff59c0b3b60 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.713+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c0716c0 con 0x7ff59c0b3b60 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.714+0000 7ff5927fc700 1 -- 172.21.15.7:0/3660052398 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5980119e0 con 0x7ff59c0b3b60 2023-10-07T07:59:47.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.714+0000 7ff5927fc700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff580066f30 0x7ff5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:47.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.714+0000 7ff5927fc700 1 -- 172.21.15.7:0/3660052398 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff5980859b0 con 0x7ff59c0b3b60 2023-10-07T07:59:47.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.714+0000 7ff5a1d5e700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff580066f30 0x7ff5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:47.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.715+0000 7ff5a1d5e700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff580066f30 0x7ff5800693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0093f0 tx=0x7ff58c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:47.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.719+0000 7ff5927fc700 1 -- 172.21.15.7:0/3660052398 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff598052f70 con 0x7ff59c0b3b60 2023-10-07T07:59:47.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.925+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff59c0b34d0 con 0x7ff59c0b3b60 2023-10-07T07:59:47.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.926+0000 7ff5927fc700 1 -- 172.21.15.7:0/3660052398 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff598056590 con 0x7ff59c0b3b60 2023-10-07T07:59:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:47.927 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2023-10-07T07:59:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-10-07T07:59:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2023-10-07T07:59:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2023-10-07T07:59:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:47.927 INFO:teuthology.orchestra.run.smithi007.stdout:] 2023-10-07T07:59:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff580066f30 msgr2=0x7ff5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff580066f30 0x7ff5800693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0093f0 tx=0x7ff58c008040 comp rx=0 tx=0).stop 2023-10-07T07:59:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0b3b60 msgr2=0x7ff59c1502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0b3b60 0x7ff59c1502b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff59c06f850 tx=0x7ff598017000 comp rx=0 tx=0).stop 2023-10-07T07:59:47.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 shutdown_connections 2023-10-07T07:59:47.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff580066f30 0x7ff5800693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff59c159380 0x7ff59c14cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0b3b60 0x7ff59c1502b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 --2- 172.21.15.7:0/3660052398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff59c0d0250 0x7ff59c14c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:47.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 >> 172.21.15.7:0/3660052398 conn(0x7ff59c067be0 msgr2=0x7ff59c067fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:47.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 shutdown_connections 2023-10-07T07:59:47.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:47.928+0000 7ff5a47c3700 1 -- 172.21.15.7:0/3660052398 wait complete. 2023-10-07T07:59:47.943 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-07T07:59:47.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-07T07:59:48.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.078+0000 7fbacb3bc700 1 Processor -- start 2023-10-07T07:59:48.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.078+0000 7fbacb3bc700 1 -- start start 2023-10-07T07:59:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbacb3bc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 0x7fbac415a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbacb3bc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbac40b26d0 0x7fbac415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbacb3bc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbac415d490 0x7fbac414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbacb3bc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac4159fd0 con 0x7fbac40b26d0 2023-10-07T07:59:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbacb3bc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac415da10 con 0x7fbac415d490 2023-10-07T07:59:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbacb3bc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac415dcf0 con 0x7fbac40b2300 2023-10-07T07:59:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac8957700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbac40b26d0 0x7fbac415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac8957700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbac40b26d0 0x7fbac415cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47696/0 (socket says 172.21.15.7:47696) 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac8957700 1 -- 172.21.15.7:0/284467047 learned_addr learned my addr 172.21.15.7:0/284467047 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac9959700 1 --2- 172.21.15.7:0/284467047 >> v2:172.21.15.172:3300/0 conn(0x7fbac415d490 0x7fbac414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac9158700 1 --2- 172.21.15.7:0/284467047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 0x7fbac415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac8957700 1 -- 172.21.15.7:0/284467047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 msgr2=0x7fbac415a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac8957700 1 --2- 172.21.15.7:0/284467047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 0x7fbac415a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac8957700 1 -- 172.21.15.7:0/284467047 >> v2:172.21.15.172:3300/0 conn(0x7fbac415d490 msgr2=0x7fbac414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac8957700 1 --2- 172.21.15.7:0/284467047 >> v2:172.21.15.172:3300/0 conn(0x7fbac415d490 0x7fbac414fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.079+0000 7fbac8957700 1 -- 172.21.15.7:0/284467047 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac4159c60 con 0x7fbac40b26d0 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbac8957700 1 --2- 172.21.15.7:0/284467047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 0x7fbac415cf50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbac40b5ec0 tx=0x7fbac0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28f136d744fbc90 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbabbfff700 1 -- 172.21.15.7:0/284467047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac000b640 con 0x7fbac40b26d0 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbabbfff700 1 -- 172.21.15.7:0/284467047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac000b7a0 con 0x7fbac40b26d0 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbabbfff700 1 -- 172.21.15.7:0/284467047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac000bb60 con 0x7fbac40b26d0 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbacb3bc700 1 -- 172.21.15.7:0/284467047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 msgr2=0x7fbac415cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/284467047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 0x7fbac415cf50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbac40b5ec0 tx=0x7fbac0009250 comp rx=0 tx=0).stop 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbacb3bc700 1 -- 172.21.15.7:0/284467047 shutdown_connections 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/284467047 >> v2:172.21.15.172:3300/0 conn(0x7fbac415d490 0x7fbac414fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/284467047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 0x7fbac415a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/284467047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 0x7fbac415cf50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.080+0000 7fbacb3bc700 1 -- 172.21.15.7:0/284467047 >> 172.21.15.7:0/284467047 conn(0x7fbac4069e90 msgr2=0x7fbac406e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.081+0000 7fbacb3bc700 1 -- 172.21.15.7:0/284467047 shutdown_connections 2023-10-07T07:59:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.081+0000 7fbacb3bc700 1 -- 172.21.15.7:0/284467047 wait complete. 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.082+0000 7fbacb3bc700 1 Processor -- start 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.082+0000 7fbacb3bc700 1 -- start start 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.083+0000 7fbacb3bc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 0x7fbac4155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.083+0000 7fbacb3bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 0x7fbac4169700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.083+0000 7fbacb3bc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac415d490 0x7fbac40ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.083+0000 7fbacb3bc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac40c8110 con 0x7fbac40b26d0 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.083+0000 7fbacb3bc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac40c7260 con 0x7fbac415d490 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.083+0000 7fbacb3bc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac41560b0 con 0x7fbac40b2300 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac8957700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 0x7fbac4169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac8957700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 0x7fbac4169700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47702/0 (socket says 172.21.15.7:47702) 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac8957700 1 -- 172.21.15.7:0/4132355649 learned_addr learned my addr 172.21.15.7:0/4132355649 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac9158700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 0x7fbac4155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac9959700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac415d490 0x7fbac40ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac8957700 1 -- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 msgr2=0x7fbac4155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac8957700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 0x7fbac4155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac8957700 1 -- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac415d490 msgr2=0x7fbac40ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac8957700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac415d490 0x7fbac40ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac8957700 1 -- 172.21.15.7:0/4132355649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00094f0 con 0x7fbac40b26d0 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbac8957700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 0x7fbac4169700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbac00094c0 tx=0x7fbac0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbab97fa700 1 -- 172.21.15.7:0/4132355649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0000c90 con 0x7fbac40b26d0 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbab97fa700 1 -- 172.21.15.7:0/4132355649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac000b910 con 0x7fbac40b26d0 2023-10-07T07:59:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac40c7500 con 0x7fbac40b26d0 2023-10-07T07:59:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbab97fa700 1 -- 172.21.15.7:0/4132355649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0026430 con 0x7fbac40b26d0 2023-10-07T07:59:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.084+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac40c7ac0 con 0x7fbac40b26d0 2023-10-07T07:59:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.086+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac40c8280 con 0x7fbac40b26d0 2023-10-07T07:59:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.086+0000 7fbab97fa700 1 -- 172.21.15.7:0/4132355649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbac0023030 con 0x7fbac40b26d0 2023-10-07T07:59:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.086+0000 7fbab97fa700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbaa8066f30 0x7fbaa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.087+0000 7fbab97fa700 1 -- 172.21.15.7:0/4132355649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbac0085990 con 0x7fbac40b26d0 2023-10-07T07:59:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.087+0000 7fbac9158700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbaa8066f30 0x7fbaa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.087+0000 7fbac9158700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbaa8066f30 0x7fbaa80693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009db0 tx=0x7fbab40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.091+0000 7fbab97fa700 1 -- 172.21.15.7:0/4132355649 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbac0052f30 con 0x7fbac40b26d0 2023-10-07T07:59:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.294+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbac40716c0 con 0x7fbac40b26d0 2023-10-07T07:59:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.352+0000 7fbab97fa700 1 -- 172.21.15.7:0/4132355649 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbac0056550 con 0x7fbac40b26d0 2023-10-07T07:59:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbaa8066f30 msgr2=0x7fbaa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbaa8066f30 0x7fbaa80693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009db0 tx=0x7fbab40091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 msgr2=0x7fbac4169700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 0x7fbac4169700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbac00094c0 tx=0x7fbac0017000 comp rx=0 tx=0).stop 2023-10-07T07:59:48.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 shutdown_connections 2023-10-07T07:59:48.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fbaa8066f30 0x7fbaa80693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac415d490 0x7fbac40ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbac40b2300 0x7fbac4155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 --2- 172.21.15.7:0/4132355649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbac40b26d0 0x7fbac4169700 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbac00094c0 tx=0x7fbac0017000 comp rx=0 tx=0).stop 2023-10-07T07:59:48.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.357+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 >> 172.21.15.7:0/4132355649 conn(0x7fbac4069e90 msgr2=0x7fbac4156af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.358+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 shutdown_connections 2023-10-07T07:59:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.359+0000 7fbacb3bc700 1 -- 172.21.15.7:0/4132355649 wait complete. 2023-10-07T07:59:48.361 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2023-10-07T07:59:48.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-07T07:59:48.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.513+0000 7faee483b700 1 Processor -- start 2023-10-07T07:59:48.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.513+0000 7faee483b700 1 -- start start 2023-10-07T07:59:48.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faee483b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faee0067240 0x7faee015a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faee483b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faee0067610 0x7faee0153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faee483b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faee01539c0 0x7faee00aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faee483b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee006a500 con 0x7faee0067240 2023-10-07T07:59:48.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faee483b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee006f320 con 0x7faee0067610 2023-10-07T07:59:48.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faee483b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee0153e00 con 0x7faee01539c0 2023-10-07T07:59:48.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faede59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faee0067240 0x7faee015a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faede59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faee0067240 0x7faee015a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47704/0 (socket says 172.21.15.7:47704) 2023-10-07T07:59:48.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faede59c700 1 -- 172.21.15.7:0/4282808990 learned_addr learned my addr 172.21.15.7:0/4282808990 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:48.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faeddd9b700 1 --2- 172.21.15.7:0/4282808990 >> v2:172.21.15.172:3300/0 conn(0x7faee0067610 0x7faee0153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.514+0000 7faede59c700 1 -- 172.21.15.7:0/4282808990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faee01539c0 msgr2=0x7faee00aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:48.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faede59c700 1 --2- 172.21.15.7:0/4282808990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faee01539c0 0x7faee00aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faede59c700 1 -- 172.21.15.7:0/4282808990 >> v2:172.21.15.172:3300/0 conn(0x7faee0067610 msgr2=0x7faee0153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:48.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faede59c700 1 --2- 172.21.15.7:0/4282808990 >> v2:172.21.15.172:3300/0 conn(0x7faee0067610 0x7faee0153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faede59c700 1 -- 172.21.15.7:0/4282808990 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee006e7a0 con 0x7faee0067240 2023-10-07T07:59:48.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faede59c700 1 --2- 172.21.15.7:0/4282808990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 0x7faee015a2d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faee006f930 tx=0x7faed0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0a8fbda09f9eab server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:48.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faedd59a700 1 -- 172.21.15.7:0/4282808990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed000b860 con 0x7faee0067240 2023-10-07T07:59:48.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faedd59a700 1 -- 172.21.15.7:0/4282808990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed000b9c0 con 0x7faee0067240 2023-10-07T07:59:48.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faedd59a700 1 -- 172.21.15.7:0/4282808990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed000bd60 con 0x7faee0067240 2023-10-07T07:59:48.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faee483b700 1 -- 172.21.15.7:0/4282808990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 msgr2=0x7faee015a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:48.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.515+0000 7faee483b700 1 --2- 172.21.15.7:0/4282808990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 0x7faee015a2d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faee006f930 tx=0x7faed0009250 comp rx=0 tx=0).stop 2023-10-07T07:59:48.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.516+0000 7faee483b700 1 -- 172.21.15.7:0/4282808990 shutdown_connections 2023-10-07T07:59:48.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.516+0000 7faee483b700 1 --2- 172.21.15.7:0/4282808990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faee01539c0 0x7faee00aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.516+0000 7faee483b700 1 --2- 172.21.15.7:0/4282808990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 0x7faee015a2d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.516+0000 7faee483b700 1 --2- 172.21.15.7:0/4282808990 >> v2:172.21.15.172:3300/0 conn(0x7faee0067610 0x7faee0153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.516+0000 7faee483b700 1 -- 172.21.15.7:0/4282808990 >> 172.21.15.7:0/4282808990 conn(0x7faee0068100 msgr2=0x7faee0068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:48.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.516+0000 7faee483b700 1 -- 172.21.15.7:0/4282808990 shutdown_connections 2023-10-07T07:59:48.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.516+0000 7faee483b700 1 -- 172.21.15.7:0/4282808990 wait complete. 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.517+0000 7faee483b700 1 Processor -- start 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.517+0000 7faee483b700 1 -- start start 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faee483b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 0x7faee015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faee483b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faee0067610 0x7faee01501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faee483b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faee01539c0 0x7faee00c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faee483b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee016a570 con 0x7faee0067240 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faee483b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee0169c40 con 0x7faee0067610 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faee483b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee0169f20 con 0x7faee01539c0 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faede59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 0x7faee015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faede59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 0x7faee015ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47708/0 (socket says 172.21.15.7:47708) 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faede59c700 1 -- 172.21.15.7:0/508807621 learned_addr learned my addr 172.21.15.7:0/508807621 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faeddd9b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faee0067610 0x7faee01501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faeded9d700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faee01539c0 0x7faee00c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faede59c700 1 -- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faee01539c0 msgr2=0x7faee00c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faede59c700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faee01539c0 0x7faee00c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faede59c700 1 -- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faee0067610 msgr2=0x7faee01501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faede59c700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faee0067610 0x7faee01501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faede59c700 1 -- 172.21.15.7:0/508807621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00094f0 con 0x7faee0067240 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faeddd9b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faee0067610 0x7faee01501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.518+0000 7faede59c700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 0x7faee015ea80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faed0009290 tx=0x7faed0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.519+0000 7faed67fc700 1 -- 172.21.15.7:0/508807621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0026480 con 0x7faee0067240 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.519+0000 7faed67fc700 1 -- 172.21.15.7:0/508807621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed001f070 con 0x7faee0067240 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.519+0000 7faed67fc700 1 -- 172.21.15.7:0/508807621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0026830 con 0x7faee0067240 2023-10-07T07:59:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.519+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee016a200 con 0x7faee0067240 2023-10-07T07:59:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.519+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee01698b0 con 0x7faee0067240 2023-10-07T07:59:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.520+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee016a850 con 0x7faee0067240 2023-10-07T07:59:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.521+0000 7faed67fc700 1 -- 172.21.15.7:0/508807621 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faed0026ba0 con 0x7faee0067240 2023-10-07T07:59:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.521+0000 7faed67fc700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faec0066f30 0x7faec00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.521+0000 7faed67fc700 1 -- 172.21.15.7:0/508807621 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5223+0+0 (secure 0 0 0) 0x7faed0021070 con 0x7faee0067240 2023-10-07T07:59:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.522+0000 7faeddd9b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faec0066f30 0x7faec00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.522+0000 7faeddd9b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faec0066f30 0x7faec00693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faee015efc0 tx=0x7faec8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.525+0000 7faed67fc700 1 -- 172.21.15.7:0/508807621 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faed0053970 con 0x7faee0067240 2023-10-07T07:59:48.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:48.728+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7faee004c3c0 con 0x7faee0067240 2023-10-07T07:59:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.359+0000 7faed67fc700 1 -- 172.21.15.7:0/508807621 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7faed0056f90 con 0x7faee0067240 2023-10-07T07:59:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faec0066f30 msgr2=0x7faec00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faec0066f30 0x7faec00693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faee015efc0 tx=0x7faec8008040 comp rx=0 tx=0).stop 2023-10-07T07:59:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 msgr2=0x7faee015ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 0x7faee015ea80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faed0009290 tx=0x7faed0017000 comp rx=0 tx=0).stop 2023-10-07T07:59:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 shutdown_connections 2023-10-07T07:59:49.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faec0066f30 0x7faec00693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:49.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faee01539c0 0x7faee00c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:49.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee0067240 0x7faee015ea80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:49.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 --2- 172.21.15.7:0/508807621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faee0067610 0x7faee01501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:49.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.370+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 >> 172.21.15.7:0/508807621 conn(0x7faee0068100 msgr2=0x7faee0151e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.374+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 shutdown_connections 2023-10-07T07:59:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.375+0000 7faee483b700 1 -- 172.21.15.7:0/508807621 wait complete. 2023-10-07T07:59:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2023-10-07T07:59:49.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-07T07:59:49.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.533+0000 7f425d1d7700 1 Processor -- start 2023-10-07T07:59:49.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.533+0000 7f425d1d7700 1 -- start start 2023-10-07T07:59:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.534+0000 7f425d1d7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4258155d90 0x7f4258156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.534+0000 7f425d1d7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42580c7a80 0x7f425806c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.534+0000 7f425d1d7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425806b6f0 con 0x7f42580c75e0 2023-10-07T07:59:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.534+0000 7f425d1d7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42580cbbf0 con 0x7f42580c7a80 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.534+0000 7f425d1d7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425806bf70 con 0x7f4258155d90 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.534+0000 7f425759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42580c7a80 0x7f425806c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.534+0000 7f425759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42580c7a80 0x7f425806c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38418/0 (socket says 172.21.15.7:38418) 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.534+0000 7f425759e700 1 -- 172.21.15.7:0/1780156552 learned_addr learned my addr 172.21.15.7:0/1780156552 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.535+0000 7f4256d9d700 1 --2- 172.21.15.7:0/1780156552 >> v2:172.21.15.172:3300/0 conn(0x7f4258155d90 0x7f4258156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.535+0000 7f425759e700 1 -- 172.21.15.7:0/1780156552 >> v2:172.21.15.172:3300/0 conn(0x7f4258155d90 msgr2=0x7f4258156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.535+0000 7f425759e700 1 --2- 172.21.15.7:0/1780156552 >> v2:172.21.15.172:3300/0 conn(0x7f4258155d90 0x7f4258156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.535+0000 7f425759e700 1 -- 172.21.15.7:0/1780156552 >> v1:172.21.15.172:6789/0 conn(0x7f42580c75e0 legacy=0x7f425806e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.535+0000 7f425759e700 1 -- 172.21.15.7:0/1780156552 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42580cb910 con 0x7f42580c7a80 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.535+0000 7f425759e700 1 --2- 172.21.15.7:0/1780156552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580c7a80 0x7f425806c400 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f425806f370 tx=0x7f425000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af69a705c518983 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.535+0000 7f4255d9b700 1 -- 172.21.15.7:0/1780156552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4250010070 con 0x7f42580c7a80 2023-10-07T07:59:49.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.535+0000 7f4255d9b700 1 -- 172.21.15.7:0/1780156552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425000dde0 con 0x7f42580c7a80 2023-10-07T07:59:49.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.535+0000 7f4255d9b700 1 -- 172.21.15.7:0/1780156552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000b320 con 0x7f42580c7a80 2023-10-07T07:59:49.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.536+0000 7f425d1d7700 1 -- 172.21.15.7:0/1780156552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580c7a80 msgr2=0x7f425806c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.536+0000 7f425d1d7700 1 --2- 172.21.15.7:0/1780156552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580c7a80 0x7f425806c400 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f425806f370 tx=0x7f425000ba50 comp rx=0 tx=0).stop 2023-10-07T07:59:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.536+0000 7f425d1d7700 1 -- 172.21.15.7:0/1780156552 shutdown_connections 2023-10-07T07:59:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.536+0000 7f425d1d7700 1 --2- 172.21.15.7:0/1780156552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580c7a80 0x7f425806c400 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.536+0000 7f425d1d7700 1 --2- 172.21.15.7:0/1780156552 >> v2:172.21.15.172:3300/0 conn(0x7f4258155d90 0x7f4258156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.536+0000 7f425d1d7700 1 -- 172.21.15.7:0/1780156552 >> 172.21.15.7:0/1780156552 conn(0x7f4258068100 msgr2=0x7f4258068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.536+0000 7f425d1d7700 1 -- 172.21.15.7:0/1780156552 shutdown_connections 2023-10-07T07:59:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.536+0000 7f425d1d7700 1 -- 172.21.15.7:0/1780156552 wait complete. 2023-10-07T07:59:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.537+0000 7f425d1d7700 1 Processor -- start 2023-10-07T07:59:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.537+0000 7f425d1d7700 1 -- start start 2023-10-07T07:59:49.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425d1d7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42580c75e0 0x7f4258146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:49.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425d1d7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42580c7a80 0x7f42581587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:49.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425d1d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4258155d90 0x7f4258158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:49.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425d1d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258147020 con 0x7f4258155d90 2023-10-07T07:59:49.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425d1d7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42580af140 con 0x7f42580c7a80 2023-10-07T07:59:49.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425d1d7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42580b2d10 con 0x7f42580c75e0 2023-10-07T07:59:49.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4258155d90 0x7f4258158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:49.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4258155d90 0x7f4258158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38434/0 (socket says 172.21.15.7:38434) 2023-10-07T07:59:49.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425759e700 1 -- 172.21.15.7:0/586375010 learned_addr learned my addr 172.21.15.7:0/586375010 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:49.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f4256d9d700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42580c75e0 0x7f4258146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.538+0000 7f425659c700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42580c7a80 0x7f42581587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f425759e700 1 -- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42580c75e0 msgr2=0x7f4258146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f425759e700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42580c75e0 0x7f4258146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f425759e700 1 -- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42580c7a80 msgr2=0x7f42581587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f425759e700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42580c7a80 0x7f42581587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f425759e700 1 -- 172.21.15.7:0/586375010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425000bcf0 con 0x7f4258155d90 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f4256d9d700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42580c75e0 0x7f4258146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f425759e700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4258155d90 0x7f4258158d00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f42580c4000 tx=0x7f425001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f424effd700 1 -- 172.21.15.7:0/586375010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000b4d0 con 0x7f4258155d90 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f424effd700 1 -- 172.21.15.7:0/586375010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425001f630 con 0x7f4258155d90 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42580ae530 con 0x7f4258155d90 2023-10-07T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f424effd700 1 -- 172.21.15.7:0/586375010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425001f9d0 con 0x7f4258155d90 2023-10-07T07:59:49.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.539+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42580aeaf0 con 0x7f4258155d90 2023-10-07T07:59:49.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.540+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42580af2b0 con 0x7f4258155d90 2023-10-07T07:59:49.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.541+0000 7f424effd700 1 -- 172.21.15.7:0/586375010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4250003040 con 0x7f4258155d90 2023-10-07T07:59:49.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.541+0000 7f424effd700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4238066f30 0x7f42380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:49.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.541+0000 7f424effd700 1 -- 172.21.15.7:0/586375010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f4250088740 con 0x7f4258155d90 2023-10-07T07:59:49.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.542+0000 7f4256d9d700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4238066f30 0x7f42380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:49.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.542+0000 7f4256d9d700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4238066f30 0x7f42380693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4240006f00 tx=0x7f4240008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:49.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.546+0000 7f424effd700 1 -- 172.21.15.7:0/586375010 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4250055a10 con 0x7f4258155d90 2023-10-07T07:59:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:49.795+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f425804c3c0 con 0x7f4258155d90 2023-10-07T07:59:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.373+0000 7f424effd700 1 -- 172.21.15.7:0/586375010 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4250059030 con 0x7f4258155d90 2023-10-07T07:59:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4238066f30 msgr2=0x7f42380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4238066f30 0x7f42380693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4240006f00 tx=0x7f4240008040 comp rx=0 tx=0).stop 2023-10-07T07:59:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4258155d90 msgr2=0x7f4258158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4258155d90 0x7f4258158d00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f42580c4000 tx=0x7f425001a000 comp rx=0 tx=0).stop 2023-10-07T07:59:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 shutdown_connections 2023-10-07T07:59:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4238066f30 0x7f42380693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4258155d90 0x7f4258158d00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42580c75e0 0x7f4258146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 --2- 172.21.15.7:0/586375010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42580c7a80 0x7f42581587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 >> 172.21.15.7:0/586375010 conn(0x7f4258068100 msgr2=0x7f42580684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 shutdown_connections 2023-10-07T07:59:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.397+0000 7f425d1d7700 1 -- 172.21.15.7:0/586375010 wait complete. 2023-10-07T07:59:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 6 and data pool 7 2023-10-07T07:59:50.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:59:50.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.550+0000 7f7bd4fdc700 1 Processor -- start 2023-10-07T07:59:50.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.550+0000 7f7bd4fdc700 1 -- start start 2023-10-07T07:59:50.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.551+0000 7f7bd4fdc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd01562f0 0x7f7bd00c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.551+0000 7f7bd4fdc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd006b700 con 0x7f7bd0155f20 2023-10-07T07:59:50.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.551+0000 7f7bd4fdc700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd006aaf0 con 0x7f7bd006e9d0 2023-10-07T07:59:50.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.551+0000 7f7bd4fdc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd006add0 con 0x7f7bd01562f0 2023-10-07T07:59:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.551+0000 7f7bce59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7bd0155f20 0x7f7bd006e2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34568/0 (socket says 172.21.15.7:34568) 2023-10-07T07:59:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.551+0000 7f7bce59c700 1 -- 172.21.15.7:0/1463353337 learned_addr learned my addr 172.21.15.7:0/1463353337 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.551+0000 7f7bced9d700 1 --2- 172.21.15.7:0/1463353337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd01562f0 0x7f7bd00c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.551+0000 7f7bcd59a700 1 -- 172.21.15.7:0/1463353337 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842331587 0 0) 0x7f7bd006b700 con 0x7f7bd0155f20 2023-10-07T07:59:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.551+0000 7f7bcd59a700 1 -- 172.21.15.7:0/1463353337 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb8003570 con 0x7f7bd0155f20 2023-10-07T07:59:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bcd59a700 1 -- 172.21.15.7:0/1463353337 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18313 0 0) 0x7f7bd006aaf0 con 0x7f7bd006e9d0 2023-10-07T07:59:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bcd59a700 1 -- 172.21.15.7:0/1463353337 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bd006b700 con 0x7f7bd006e9d0 2023-10-07T07:59:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bcd59a700 1 -- 172.21.15.7:0/1463353337 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 120593791 0 0) 0x7f7bb8003570 con 0x7f7bd0155f20 2023-10-07T07:59:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bced9d700 1 -- 172.21.15.7:0/1463353337 >> v1:172.21.15.172:6789/0 conn(0x7f7bd006e9d0 legacy=0x7f7bd00caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bced9d700 1 -- 172.21.15.7:0/1463353337 >> v1:172.21.15.7:6789/0 conn(0x7f7bd0155f20 legacy=0x7f7bd006e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bced9d700 1 -- 172.21.15.7:0/1463353337 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd006b0b0 con 0x7f7bd01562f0 2023-10-07T07:59:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bced9d700 1 --2- 172.21.15.7:0/1463353337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd01562f0 0x7f7bd00c92b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009e90 tx=0x7f7bc8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=701ae5c3ab5b303f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bcd59a700 1 -- 172.21.15.7:0/1463353337 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800e070 con 0x7f7bd01562f0 2023-10-07T07:59:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bcd59a700 1 -- 172.21.15.7:0/1463353337 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc800b870 con 0x7f7bd01562f0 2023-10-07T07:59:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.552+0000 7f7bcd59a700 1 -- 172.21.15.7:0/1463353337 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0001470 con 0x7f7bd01562f0 2023-10-07T07:59:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.553+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/1463353337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd01562f0 msgr2=0x7f7bd00c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.553+0000 7f7bd4fdc700 1 --2- 172.21.15.7:0/1463353337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd01562f0 0x7f7bd00c92b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009e90 tx=0x7f7bc8009780 comp rx=0 tx=0).stop 2023-10-07T07:59:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.553+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/1463353337 shutdown_connections 2023-10-07T07:59:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.553+0000 7f7bd4fdc700 1 --2- 172.21.15.7:0/1463353337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd01562f0 0x7f7bd00c92b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.553+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/1463353337 >> 172.21.15.7:0/1463353337 conn(0x7f7bd0068bd0 msgr2=0x7f7bd00ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.553+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/1463353337 shutdown_connections 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.553+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/1463353337 wait complete. 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.554+0000 7f7bd4fdc700 1 Processor -- start 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.554+0000 7f7bd4fdc700 1 -- start start 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bd4fdc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd006e9d0 0x7f7bd00aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bd4fdc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd0155f20 0x7f7bd00c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bd4fdc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd01562f0 0x7f7bd00c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bd4fdc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd016a790 con 0x7f7bd01562f0 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bd4fdc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd00c2720 con 0x7f7bd006e9d0 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bd4fdc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd00af400 con 0x7f7bd0155f20 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bced9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd01562f0 0x7f7bd00c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bced9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd01562f0 0x7f7bd00c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38444/0 (socket says 172.21.15.7:38444) 2023-10-07T07:59:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bced9d700 1 -- 172.21.15.7:0/2481267258 learned_addr learned my addr 172.21.15.7:0/2481267258 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bce59c700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd006e9d0 0x7f7bd00aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bcdd9b700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd0155f20 0x7f7bd00c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bced9d700 1 -- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd0155f20 msgr2=0x7f7bd00c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bced9d700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd0155f20 0x7f7bd00c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bced9d700 1 -- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd006e9d0 msgr2=0x7f7bd00aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bced9d700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd006e9d0 0x7f7bd00aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.555+0000 7f7bced9d700 1 -- 172.21.15.7:0/2481267258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc8009a20 con 0x7f7bd01562f0 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.556+0000 7f7bced9d700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd01562f0 0x7f7bd00c6620 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7bd006de70 tx=0x7f7bc8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.556+0000 7f7bbe7fc700 1 -- 172.21.15.7:0/2481267258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8012be0 con 0x7f7bd01562f0 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.556+0000 7f7bbe7fc700 1 -- 172.21.15.7:0/2481267258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc8012d40 con 0x7f7bd01562f0 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.556+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd00c1df0 con 0x7f7bd01562f0 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.556+0000 7f7bbe7fc700 1 -- 172.21.15.7:0/2481267258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc801e6a0 con 0x7f7bd01562f0 2023-10-07T07:59:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.556+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd00c23b0 con 0x7f7bd01562f0 2023-10-07T07:59:50.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.557+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd00cc350 con 0x7f7bd01562f0 2023-10-07T07:59:50.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.557+0000 7f7bbe7fc700 1 -- 172.21.15.7:0/2481267258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bc801ead0 con 0x7f7bd01562f0 2023-10-07T07:59:50.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.558+0000 7f7bbe7fc700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bb0066f30 0x7f7bb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.558+0000 7f7bbe7fc700 1 -- 172.21.15.7:0/2481267258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7bc8086370 con 0x7f7bd01562f0 2023-10-07T07:59:50.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.558+0000 7f7bce59c700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bb0066f30 0x7f7bb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.559+0000 7f7bce59c700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bb0066f30 0x7f7bb00693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00c9650 tx=0x7f7bc0001620 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:50.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.562+0000 7f7bbe7fc700 1 -- 172.21.15.7:0/2481267258 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bc80535e0 con 0x7f7bd01562f0 2023-10-07T07:59:50.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.768+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bd00716c0 con 0x7f7bd01562f0 2023-10-07T07:59:50.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.769+0000 7f7bbe7fc700 1 -- 172.21.15.7:0/2481267258 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f7bc8056c00 con 0x7f7bd01562f0 2023-10-07T07:59:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:50.771 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:59:49.796989+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:59:48.327048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T07:59:48.760594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:59:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bb0066f30 msgr2=0x7f7bb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bb0066f30 0x7f7bb00693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00c9650 tx=0x7f7bc0001620 comp rx=0 tx=0).stop 2023-10-07T07:59:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd01562f0 msgr2=0x7f7bd00c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd01562f0 0x7f7bd00c6620 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7bd006de70 tx=0x7f7bc8017000 comp rx=0 tx=0).stop 2023-10-07T07:59:50.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 shutdown_connections 2023-10-07T07:59:50.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7bb0066f30 0x7f7bb00693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd01562f0 0x7f7bd00c6620 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd006e9d0 0x7f7bd00aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 --2- 172.21.15.7:0/2481267258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd0155f20 0x7f7bd00c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 >> 172.21.15.7:0/2481267258 conn(0x7f7bd0068bd0 msgr2=0x7f7bd0155070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 shutdown_connections 2023-10-07T07:59:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.772+0000 7f7bd4fdc700 1 -- 172.21.15.7:0/2481267258 wait complete. 2023-10-07T07:59:50.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:59:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.918+0000 7f3cc50a7700 1 Processor -- start 2023-10-07T07:59:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.918+0000 7f3cc50a7700 1 -- start start 2023-10-07T07:59:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cc50a7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3cc00aaa50 0x7f3cc0164110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cc50a7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc006bb70 con 0x7f3cc01592c0 2023-10-07T07:59:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cc50a7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc00cda10 con 0x7f3cc0159690 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cc50a7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc006af60 con 0x7f3cc00aaa50 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3cc01592c0 0x7f3cc006a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34574/0 (socket says 172.21.15.7:34574) 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbed9d700 1 -- 172.21.15.7:0/3438941090 learned_addr learned my addr 172.21.15.7:0/3438941090 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbf59e700 1 --2- 172.21.15.7:0/3438941090 >> v2:172.21.15.172:3300/0 conn(0x7f3cc00aaa50 0x7f3cc0164110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922686725 0 0) 0x7f3cc006bb70 con 0x7f3cc01592c0 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cac003570 con 0x7f3cc01592c0 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2027950141 0 0) 0x7f3cac003570 con 0x7f3cc01592c0 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 >> v2:172.21.15.172:3300/0 conn(0x7f3cc00aaa50 msgr2=0x7f3cc0164110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbdd9b700 1 --2- 172.21.15.7:0/3438941090 >> v2:172.21.15.172:3300/0 conn(0x7f3cc00aaa50 0x7f3cc0164110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 >> v1:172.21.15.172:6789/0 conn(0x7f3cc0159690 legacy=0x7f3cc00cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc006b240 con 0x7f3cc01592c0 2023-10-07T07:59:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f3ca8001470 con 0x7f3cc01592c0 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.919+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 >> v1:172.21.15.7:6789/0 conn(0x7f3cc01592c0 legacy=0x7f3cc006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbdd9b700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3cac003a60 0x7f3cac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbdd9b700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac004810 0x7f3cac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbdd9b700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac0051c0 0x7f3cac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc006af60 con 0x7f3cac004810 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac00c0d0 con 0x7f3cac0051c0 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac00c290 con 0x7f3cac003a60 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbe59c700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac004810 0x7f3cac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbf59e700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac0051c0 0x7f3cac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbed9d700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3cac003a60 0x7f3cac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbe59c700 1 -- 172.21.15.7:0/3438941090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3cac003a60 msgr2=0x7f3cac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbe59c700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3cac003a60 0x7f3cac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbe59c700 1 -- 172.21.15.7:0/3438941090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac0051c0 msgr2=0x7f3cac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbe59c700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac0051c0 0x7f3cac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbe59c700 1 -- 172.21.15.7:0/3438941090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac003570 con 0x7f3cac004810 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbf59e700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac0051c0 0x7f3cac00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbf59e700 1 -- 172.21.15.7:0/3438941090 reap_dead start 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbe59c700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac004810 0x7f3cac004c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00018b0 tx=0x7f3cb0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd3fbb6191793b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0012420 con 0x7f3cac004810 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb00125a0 con 0x7f3cac004810 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.920+0000 7f3cbdd9b700 1 -- 172.21.15.7:0/3438941090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0012960 con 0x7f3cac004810 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.921+0000 7f3cc50a7700 1 -- 172.21.15.7:0/3438941090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac004810 msgr2=0x7f3cac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.921+0000 7f3cc50a7700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac004810 0x7f3cac004c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00018b0 tx=0x7f3cb0007d30 comp rx=0 tx=0).stop 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.921+0000 7f3cc50a7700 1 -- 172.21.15.7:0/3438941090 shutdown_connections 2023-10-07T07:59:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.921+0000 7f3cc50a7700 1 --2- 172.21.15.7:0/3438941090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac004810 0x7f3cac004c80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.921+0000 7f3cc50a7700 1 -- 172.21.15.7:0/3438941090 >> 172.21.15.7:0/3438941090 conn(0x7f3cc0068bd0 msgr2=0x7f3cc00ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:50.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.921+0000 7f3cc50a7700 1 -- 172.21.15.7:0/3438941090 shutdown_connections 2023-10-07T07:59:50.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.921+0000 7f3cc50a7700 1 -- 172.21.15.7:0/3438941090 wait complete. 2023-10-07T07:59:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.922+0000 7f3cc50a7700 1 Processor -- start 2023-10-07T07:59:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.923+0000 7f3cc50a7700 1 -- start start 2023-10-07T07:59:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.923+0000 7f3cc50a7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac004810 0x7f3cc01430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.923+0000 7f3cc50a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc00ac8a0 0x7f3cc015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.923+0000 7f3cc50a7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3cc016cb20 0x7f3cc016cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.923+0000 7f3cc50a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc016d500 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.923+0000 7f3cc50a7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc016d9b0 con 0x7f3cac004810 2023-10-07T07:59:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.923+0000 7f3cc50a7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc00acd10 con 0x7f3cc016cb20 2023-10-07T07:59:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.923+0000 7f3cbe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc00ac8a0 0x7f3cc015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.923+0000 7f3cbe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc00ac8a0 0x7f3cc015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38466/0 (socket says 172.21.15.7:38466) 2023-10-07T07:59:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbe59c700 1 -- 172.21.15.7:0/887175617 learned_addr learned my addr 172.21.15.7:0/887175617 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbed9d700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac004810 0x7f3cc01430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbf59e700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3cc016cb20 0x7f3cc016cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbe59c700 1 -- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3cc016cb20 msgr2=0x7f3cc016cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbe59c700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3cc016cb20 0x7f3cc016cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbe59c700 1 -- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac004810 msgr2=0x7f3cc01430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbe59c700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac004810 0x7f3cc01430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbe59c700 1 -- 172.21.15.7:0/887175617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc00acff0 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbed9d700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac004810 0x7f3cc01430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cbe59c700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc00ac8a0 0x7f3cc015d370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000c3f0 tx=0x7f3cb0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cb6ffd700 1 -- 172.21.15.7:0/887175617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0001200 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cb6ffd700 1 -- 172.21.15.7:0/887175617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb0012cd0 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc00ad2d0 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cb6ffd700 1 -- 172.21.15.7:0/887175617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0012940 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.924+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc00ad890 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.925+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc00ada50 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.926+0000 7f3cb6ffd700 1 -- 172.21.15.7:0/887175617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cb001f620 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.926+0000 7f3cb6ffd700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3ca0066f30 0x7f3ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.926+0000 7f3cb6ffd700 1 -- 172.21.15.7:0/887175617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3cb0085f90 con 0x7f3cc00ac8a0 2023-10-07T07:59:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.926+0000 7f3cbed9d700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3ca0066f30 0x7f3ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.927+0000 7f3cbed9d700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3ca0066f30 0x7f3ca00693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8001ce0 tx=0x7f3ca8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:50.931+0000 7f3cb6ffd700 1 -- 172.21.15.7:0/887175617 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cb00b75a0 con 0x7f3cc00ac8a0 2023-10-07T07:59:51.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.131+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cc00716c0 con 0x7f3cc00ac8a0 2023-10-07T07:59:51.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.132+0000 7f3cb6ffd700 1 -- 172.21.15.7:0/887175617 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f3cb0017090 con 0x7f3cc00ac8a0 2023-10-07T07:59:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:51.133 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:59:49.796989+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:59:48.327048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T07:59:48.760594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:59:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3ca0066f30 msgr2=0x7f3ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3ca0066f30 0x7f3ca00693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8001ce0 tx=0x7f3ca8009040 comp rx=0 tx=0).stop 2023-10-07T07:59:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc00ac8a0 msgr2=0x7f3cc015d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc00ac8a0 0x7f3cc015d370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000c3f0 tx=0x7f3cb0015000 comp rx=0 tx=0).stop 2023-10-07T07:59:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 shutdown_connections 2023-10-07T07:59:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3ca0066f30 0x7f3ca00693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3cc016cb20 0x7f3cc016cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cac004810 0x7f3cc01430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 --2- 172.21.15.7:0/887175617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc00ac8a0 0x7f3cc015d370 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 >> 172.21.15.7:0/887175617 conn(0x7f3cc0068bd0 msgr2=0x7f3cc00c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:51.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 shutdown_connections 2023-10-07T07:59:51.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.134+0000 7f3cc50a7700 1 -- 172.21.15.7:0/887175617 wait complete. 2023-10-07T07:59:51.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-07T07:59:51.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.283+0000 7f9f6b218700 1 Processor -- start 2023-10-07T07:59:51.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.284+0000 7f9f6b218700 1 -- start start 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.284+0000 7f9f6b218700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067240 0x7f9f6415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.284+0000 7f9f6b218700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9f64153b90 0x7f9f640aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.284+0000 7f9f6b218700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6406f320 con 0x7f9f64067610 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.284+0000 7f9f6b218700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f6406a500 con 0x7f9f64153b90 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.284+0000 7f9f6b218700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6406e7a0 con 0x7f9f64067240 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f697b5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9f64153b90 0x7f9f640aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f697b5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9f64153b90 0x7f9f640aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38476/0 (socket says 172.21.15.7:38476) 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f697b5700 1 -- 172.21.15.7:0/2431245257 learned_addr learned my addr 172.21.15.7:0/2431245257 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f68fb4700 1 --2- 172.21.15.7:0/2431245257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067240 0x7f9f6415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:51.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f697b5700 1 -- 172.21.15.7:0/2431245257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067240 msgr2=0x7f9f6415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f697b5700 1 --2- 172.21.15.7:0/2431245257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067240 0x7f9f6415a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f697b5700 1 -- 172.21.15.7:0/2431245257 >> v1:172.21.15.172:6789/0 conn(0x7f9f64067610 legacy=0x7f9f64153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f697b5700 1 -- 172.21.15.7:0/2431245257 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6406ea80 con 0x7f9f64153b90 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f697b5700 1 --2- 172.21.15.7:0/2431245257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64153b90 0x7f9f640aa980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009f20 tx=0x7f9f5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d837febd27b2b55 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f637fe700 1 -- 172.21.15.7:0/2431245257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00e070 con 0x7f9f64153b90 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f637fe700 1 -- 172.21.15.7:0/2431245257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c00b9d0 con 0x7f9f64153b90 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.285+0000 7f9f637fe700 1 -- 172.21.15.7:0/2431245257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0092e0 con 0x7f9f64153b90 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.286+0000 7f9f6b218700 1 -- 172.21.15.7:0/2431245257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64153b90 msgr2=0x7f9f640aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.286+0000 7f9f6b218700 1 --2- 172.21.15.7:0/2431245257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64153b90 0x7f9f640aa980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009f20 tx=0x7f9f5c009840 comp rx=0 tx=0).stop 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.286+0000 7f9f6b218700 1 -- 172.21.15.7:0/2431245257 shutdown_connections 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.286+0000 7f9f6b218700 1 --2- 172.21.15.7:0/2431245257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64153b90 0x7f9f640aa980 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:51.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.286+0000 7f9f6b218700 1 --2- 172.21.15.7:0/2431245257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067240 0x7f9f6415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.286+0000 7f9f6b218700 1 -- 172.21.15.7:0/2431245257 >> 172.21.15.7:0/2431245257 conn(0x7f9f64068100 msgr2=0x7f9f64068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.286+0000 7f9f6b218700 1 -- 172.21.15.7:0/2431245257 shutdown_connections 2023-10-07T07:59:51.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.286+0000 7f9f6b218700 1 -- 172.21.15.7:0/2431245257 wait complete. 2023-10-07T07:59:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.288+0000 7f9f6b218700 1 Processor -- start 2023-10-07T07:59:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.288+0000 7f9f6b218700 1 -- start start 2023-10-07T07:59:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f6b218700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64067240 0x7f9f64149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f6b218700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067610 0x7f9f6416cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:51.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f6b218700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f64153b90 0x7f9f6416d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f6b218700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640c48b0 con 0x7f9f64067240 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f6b218700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640b1c00 con 0x7f9f64153b90 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f6b218700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640b1ee0 con 0x7f9f64067610 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f68fb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64067240 0x7f9f64149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f68fb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64067240 0x7f9f64149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38478/0 (socket says 172.21.15.7:38478) 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f68fb4700 1 -- 172.21.15.7:0/990836301 learned_addr learned my addr 172.21.15.7:0/990836301 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f63fff700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067610 0x7f9f6416cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f697b5700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f64153b90 0x7f9f6416d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f68fb4700 1 -- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067610 msgr2=0x7f9f6416cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f68fb4700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067610 0x7f9f6416cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:51.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f68fb4700 1 -- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f64153b90 msgr2=0x7f9f6416d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f68fb4700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f64153b90 0x7f9f6416d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f68fb4700 1 -- 172.21.15.7:0/990836301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c009ae0 con 0x7f9f64067240 2023-10-07T07:59:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.289+0000 7f9f68fb4700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64067240 0x7f9f64149a90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800ec80 tx=0x7f9f580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.290+0000 7f9f60ff9700 1 -- 172.21.15.7:0/990836301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800c850 con 0x7f9f64067240 2023-10-07T07:59:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.290+0000 7f9f60ff9700 1 -- 172.21.15.7:0/990836301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5800c9b0 con 0x7f9f64067240 2023-10-07T07:59:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.290+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f640b21c0 con 0x7f9f64067240 2023-10-07T07:59:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.290+0000 7f9f60ff9700 1 -- 172.21.15.7:0/990836301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800cd50 con 0x7f9f64067240 2023-10-07T07:59:51.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.290+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f64148d50 con 0x7f9f64067240 2023-10-07T07:59:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.291+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6404c3c0 con 0x7f9f64067240 2023-10-07T07:59:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.292+0000 7f9f60ff9700 1 -- 172.21.15.7:0/990836301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f58013680 con 0x7f9f64067240 2023-10-07T07:59:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.292+0000 7f9f60ff9700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f48066f30 0x7f9f480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.292+0000 7f9f60ff9700 1 -- 172.21.15.7:0/990836301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f58084760 con 0x7f9f64067240 2023-10-07T07:59:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.292+0000 7f9f63fff700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f48066f30 0x7f9f480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.293+0000 7f9f63fff700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f48066f30 0x7f9f480693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f64153750 tx=0x7f9f50007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:51.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.297+0000 7f9f60ff9700 1 -- 172.21.15.7:0/990836301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f580519d0 con 0x7f9f64067240 2023-10-07T07:59:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:51.551+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9f6406ed60 con 0x7f9f64067240 2023-10-07T07:59:52.352 INFO:tasks.ceph.mds.c.smithi007.stderr:2023-10-07T07:59:52.352+0000 7fa5cf84d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T07:59:52.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.391+0000 7f9f60ff9700 1 -- 172.21.15.7:0/990836301 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9f58054ff0 con 0x7f9f64067240 2023-10-07T07:59:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f48066f30 msgr2=0x7f9f480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f48066f30 0x7f9f480693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f64153750 tx=0x7f9f50007ec0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64067240 msgr2=0x7f9f64149a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64067240 0x7f9f64149a90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800ec80 tx=0x7f9f580091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 shutdown_connections 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9f48066f30 0x7f9f480693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f64153b90 0x7f9f6416d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f64067240 0x7f9f64149a90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 --2- 172.21.15.7:0/990836301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f64067610 0x7f9f6416cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 >> 172.21.15.7:0/990836301 conn(0x7f9f64068100 msgr2=0x7f9f640c35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 shutdown_connections 2023-10-07T07:59:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.394+0000 7f9f6b218700 1 -- 172.21.15.7:0/990836301 wait complete. 2023-10-07T07:59:52.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-07T07:59:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.550+0000 7f8583d33700 1 Processor -- start 2023-10-07T07:59:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.550+0000 7f8583d33700 1 -- start start 2023-10-07T07:59:52.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.551+0000 7f8583d33700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f857c157b20 0x7f857c157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:52.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.551+0000 7f8583d33700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c161380 con 0x7f857c15e7a0 2023-10-07T07:59:52.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.551+0000 7f8583d33700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c161660 con 0x7f857c0b0190 2023-10-07T07:59:52.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.551+0000 7f8583d33700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857c161940 con 0x7f857c157b20 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.551+0000 7f85812ce700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f857c15e7a0 0x7f857c0afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34584/0 (socket says 172.21.15.7:34584) 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.551+0000 7f85812ce700 1 -- 172.21.15.7:0/3479484801 learned_addr learned my addr 172.21.15.7:0/3479484801 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8581acf700 1 --2- 172.21.15.7:0/3479484801 >> v2:172.21.15.172:3300/0 conn(0x7f857c157b20 0x7f857c157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005848756 0 0) 0x7f857c161380 con 0x7f857c15e7a0 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8568003570 con 0x7f857c15e7a0 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489572868 0 0) 0x7f857c161660 con 0x7f857c0b0190 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f857c161380 con 0x7f857c0b0190 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3478666003 0 0) 0x7f8568003570 con 0x7f857c15e7a0 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 >> v2:172.21.15.172:3300/0 conn(0x7f857c157b20 msgr2=0x7f857c157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 --2- 172.21.15.7:0/3479484801 >> v2:172.21.15.172:3300/0 conn(0x7f857c157b20 0x7f857c157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 >> v1:172.21.15.172:6789/0 conn(0x7f857c0b0190 legacy=0x7f857c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c161c20 con 0x7f857c15e7a0 2023-10-07T07:59:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f8578001470 con 0x7f857c15e7a0 2023-10-07T07:59:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 >> v1:172.21.15.7:6789/0 conn(0x7f857c15e7a0 legacy=0x7f857c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f8568004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568004810 0x7f8568004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8568005210 0x7f856800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c161940 con 0x7f8568003c40 2023-10-07T07:59:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856800c130 con 0x7f8568004810 2023-10-07T07:59:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856800c2c0 con 0x7f8568005210 2023-10-07T07:59:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8581acf700 1 --2- 172.21.15.7:0/3479484801 >> v2:172.21.15.172:3300/0 conn(0x7f857c157b20 0x7f857c157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f85812ce700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568004810 0x7f8568004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f8581acf700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f8568004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.552+0000 7f85822d0700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8568005210 0x7f856800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f8581acf700 1 -- 172.21.15.7:0/3479484801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8568005210 msgr2=0x7f856800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f8581acf700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8568005210 0x7f856800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f8581acf700 1 -- 172.21.15.7:0/3479484801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568004810 msgr2=0x7f8568004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f8581acf700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568004810 0x7f8568004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f8581acf700 1 -- 172.21.15.7:0/3479484801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8568003570 con 0x7f8568003c40 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f85822d0700 1 -- 172.21.15.7:0/3479484801 reap_dead start 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f8581acf700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f8568004030 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f856c009de0 tx=0x7f856c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d848948c29024475 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c009740 con 0x7f8568003c40 2023-10-07T07:59:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c00e8f0 con 0x7f8568003c40 2023-10-07T07:59:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.553+0000 7f8580acd700 1 -- 172.21.15.7:0/3479484801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c00ec90 con 0x7f8568003c40 2023-10-07T07:59:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.554+0000 7f8583d33700 1 -- 172.21.15.7:0/3479484801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 msgr2=0x7f8568004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.554+0000 7f8583d33700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f8568004030 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f856c009de0 tx=0x7f856c0091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.554+0000 7f8583d33700 1 -- 172.21.15.7:0/3479484801 shutdown_connections 2023-10-07T07:59:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.554+0000 7f8583d33700 1 --2- 172.21.15.7:0/3479484801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f8568004030 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.554+0000 7f8583d33700 1 -- 172.21.15.7:0/3479484801 >> 172.21.15.7:0/3479484801 conn(0x7f857c06d350 msgr2=0x7f857c06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.554+0000 7f8583d33700 1 -- 172.21.15.7:0/3479484801 shutdown_connections 2023-10-07T07:59:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.554+0000 7f8583d33700 1 -- 172.21.15.7:0/3479484801 wait complete. 2023-10-07T07:59:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.555+0000 7f8583d33700 1 Processor -- start 2023-10-07T07:59:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.555+0000 7f8583d33700 1 -- start start 2023-10-07T07:59:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8583d33700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f857c145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8583d33700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f857c145ac0 0x7f857c0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8583d33700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f857c0ce400 0x7f857c1462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8583d33700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c0ce860 con 0x7f8568003c40 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8583d33700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c1450f0 con 0x7f857c0ce400 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8583d33700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857c1444e0 con 0x7f857c145ac0 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8581acf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f857c145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8581acf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f857c145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38502/0 (socket says 172.21.15.7:38502) 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8581acf700 1 -- 172.21.15.7:0/1762370102 learned_addr learned my addr 172.21.15.7:0/1762370102 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f85812ce700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f857c145ac0 0x7f857c0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f85822d0700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f857c0ce400 0x7f857c1462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8581acf700 1 -- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f857c145ac0 msgr2=0x7f857c0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8581acf700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f857c145ac0 0x7f857c0cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8581acf700 1 -- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f857c0ce400 msgr2=0x7f857c1462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8581acf700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f857c0ce400 0x7f857c1462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8581acf700 1 -- 172.21.15.7:0/1762370102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c1447c0 con 0x7f8568003c40 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8581acf700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f857c145580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f85680057b0 tx=0x7f856c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.556+0000 7f8571ffb700 1 -- 172.21.15.7:0/1762370102 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c00ec70 con 0x7f8568003c40 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.557+0000 7f8571ffb700 1 -- 172.21.15.7:0/1762370102 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c021040 con 0x7f8568003c40 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.557+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857c144aa0 con 0x7f8568003c40 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.557+0000 7f8571ffb700 1 -- 172.21.15.7:0/1762370102 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c025620 con 0x7f8568003c40 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.557+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c1467e0 con 0x7f8568003c40 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.558+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857c0ce800 con 0x7f8568003c40 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.558+0000 7f8571ffb700 1 -- 172.21.15.7:0/1762370102 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f856c025990 con 0x7f8568003c40 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.559+0000 7f8571ffb700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8560066f60 0x7f8560069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.559+0000 7f8571ffb700 1 -- 172.21.15.7:0/1762370102 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f856c085e00 con 0x7f8568003c40 2023-10-07T07:59:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.559+0000 7f85812ce700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8560066f60 0x7f8560069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.559+0000 7f85812ce700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8560066f60 0x7f8560069410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f857c155810 tx=0x7f857c06a2a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.563+0000 7f8571ffb700 1 -- 172.21.15.7:0/1762370102 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f857c0ce800 con 0x7f8568003c40 2023-10-07T07:59:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:52.812+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f857c0ce800 con 0x7f8568003c40 2023-10-07T07:59:53.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.392+0000 7f8571ffb700 1 -- 172.21.15.7:0/1762370102 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f856c053070 con 0x7f8568003c40 2023-10-07T07:59:53.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.395+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8560066f60 msgr2=0x7f8560069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.395+0000 7f8583d33700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8560066f60 0x7f8560069410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f857c155810 tx=0x7f857c06a2a0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 msgr2=0x7f857c145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f857c145580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f85680057b0 tx=0x7f856c019000 comp rx=0 tx=0).stop 2023-10-07T07:59:53.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 shutdown_connections 2023-10-07T07:59:53.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8560066f60 0x7f8560069410 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f857c0ce400 0x7f857c1462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568003c40 0x7f857c145580 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 --2- 172.21.15.7:0/1762370102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f857c145ac0 0x7f857c0cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 >> 172.21.15.7:0/1762370102 conn(0x7f857c06d350 msgr2=0x7f857c0afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:53.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 shutdown_connections 2023-10-07T07:59:53.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.396+0000 7f8583d33700 1 -- 172.21.15.7:0/1762370102 wait complete. 2023-10-07T07:59:53.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:59:53.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.547+0000 7f059ab4a700 1 Processor -- start 2023-10-07T07:59:53.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.547+0000 7f059ab4a700 1 -- start start 2023-10-07T07:59:53.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f059ab4a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0594159690 0x7f05940cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f059ab4a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05940cb0c0 0x7f05940a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f059ab4a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059406bb70 con 0x7f05941592c0 2023-10-07T07:59:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f059ab4a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05940cda10 con 0x7f05940cb0c0 2023-10-07T07:59:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f059ab4a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059406af60 con 0x7f0594159690 2023-10-07T07:59:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f05990e7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05940cb0c0 0x7f05940a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f05988e6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f05941592c0 0x7f059406a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34586/0 (socket says 172.21.15.7:34586) 2023-10-07T07:59:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f05988e6700 1 -- 172.21.15.7:0/2832699590 learned_addr learned my addr 172.21.15.7:0/2832699590 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f0593fff700 1 --2- 172.21.15.7:0/2832699590 >> v2:172.21.15.172:3300/0 conn(0x7f0594159690 0x7f05940cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f05990e7700 1 -- 172.21.15.7:0/2832699590 >> v2:172.21.15.172:3300/0 conn(0x7f0594159690 msgr2=0x7f05940cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f05990e7700 1 --2- 172.21.15.7:0/2832699590 >> v2:172.21.15.172:3300/0 conn(0x7f0594159690 0x7f05940cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f05990e7700 1 -- 172.21.15.7:0/2832699590 >> v1:172.21.15.7:6789/0 conn(0x7f05941592c0 legacy=0x7f059406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f05990e7700 1 -- 172.21.15.7:0/2832699590 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059406b240 con 0x7f05940cb0c0 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f05937fe700 1 -- 172.21.15.7:0/2832699590 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295722219 0 0) 0x7f059406bb70 con 0x7f05941592c0 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.548+0000 7f05990e7700 1 --2- 172.21.15.7:0/2832699590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05940cb0c0 0x7f05940a9a40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f05940caa90 tx=0x7f058c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae675104485578e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f05937fe700 1 -- 172.21.15.7:0/2832699590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00e070 con 0x7f05940cb0c0 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f05937fe700 1 -- 172.21.15.7:0/2832699590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c00bb60 con 0x7f05940cb0c0 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f05937fe700 1 -- 172.21.15.7:0/2832699590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c012430 con 0x7f05940cb0c0 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f059ab4a700 1 -- 172.21.15.7:0/2832699590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05940cb0c0 msgr2=0x7f05940a9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f059ab4a700 1 --2- 172.21.15.7:0/2832699590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05940cb0c0 0x7f05940a9a40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f05940caa90 tx=0x7f058c009780 comp rx=0 tx=0).stop 2023-10-07T07:59:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f059ab4a700 1 -- 172.21.15.7:0/2832699590 shutdown_connections 2023-10-07T07:59:53.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f059ab4a700 1 --2- 172.21.15.7:0/2832699590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05940cb0c0 0x7f05940a9a40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f059ab4a700 1 --2- 172.21.15.7:0/2832699590 >> v2:172.21.15.172:3300/0 conn(0x7f0594159690 0x7f05940cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f059ab4a700 1 -- 172.21.15.7:0/2832699590 >> 172.21.15.7:0/2832699590 conn(0x7f0594068bd0 msgr2=0x7f05940ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:53.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f059ab4a700 1 -- 172.21.15.7:0/2832699590 shutdown_connections 2023-10-07T07:59:53.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.549+0000 7f059ab4a700 1 -- 172.21.15.7:0/2832699590 wait complete. 2023-10-07T07:59:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.550+0000 7f059ab4a700 1 Processor -- start 2023-10-07T07:59:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.551+0000 7f059ab4a700 1 -- start start 2023-10-07T07:59:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.551+0000 7f059ab4a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05940cb0c0 0x7f05941554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.551+0000 7f059ab4a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05941592c0 0x7f05940ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.551+0000 7f059ab4a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594159690 0x7f05940c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.551+0000 7f059ab4a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05940c69c0 con 0x7f0594159690 2023-10-07T07:59:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.551+0000 7f059ab4a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05940c5280 con 0x7f05941592c0 2023-10-07T07:59:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.551+0000 7f059ab4a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05940ae150 con 0x7f05940cb0c0 2023-10-07T07:59:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05990e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594159690 0x7f05940c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05990e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594159690 0x7f05940c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38524/0 (socket says 172.21.15.7:38524) 2023-10-07T07:59:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05990e7700 1 -- 172.21.15.7:0/2398465557 learned_addr learned my addr 172.21.15.7:0/2398465557 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f0593fff700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05941592c0 0x7f05940ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05988e6700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05940cb0c0 0x7f05941554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05990e7700 1 -- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05940cb0c0 msgr2=0x7f05941554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05990e7700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05940cb0c0 0x7f05941554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05990e7700 1 -- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05941592c0 msgr2=0x7f05940ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05990e7700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05941592c0 0x7f05940ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05990e7700 1 -- 172.21.15.7:0/2398465557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c009a20 con 0x7f0594159690 2023-10-07T07:59:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f05990e7700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594159690 0x7f05940c6cd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0594069020 tx=0x7f058c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f0590ff9700 1 -- 172.21.15.7:0/2398465557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c027470 con 0x7f0594159690 2023-10-07T07:59:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f0590ff9700 1 -- 172.21.15.7:0/2398465557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c0275d0 con 0x7f0594159690 2023-10-07T07:59:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05941559f0 con 0x7f0594159690 2023-10-07T07:59:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f0590ff9700 1 -- 172.21.15.7:0/2398465557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c027990 con 0x7f0594159690 2023-10-07T07:59:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.552+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05940ade70 con 0x7f0594159690 2023-10-07T07:59:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.553+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05940c92c0 con 0x7f0594159690 2023-10-07T07:59:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.554+0000 7f0590ff9700 1 -- 172.21.15.7:0/2398465557 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f058c024070 con 0x7f0594159690 2023-10-07T07:59:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.554+0000 7f0590ff9700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0578066f30 0x7f05780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.554+0000 7f0590ff9700 1 -- 172.21.15.7:0/2398465557 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f058c086ec0 con 0x7f0594159690 2023-10-07T07:59:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.555+0000 7f05988e6700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0578066f30 0x7f05780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.555+0000 7f05988e6700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0578066f30 0x7f05780693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0588007fd0 tx=0x7f0588007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.559+0000 7f0590ff9700 1 -- 172.21.15.7:0/2398465557 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f058c054130 con 0x7f0594159690 2023-10-07T07:59:53.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.807+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05940ccb30 con 0x7f0594159690 2023-10-07T07:59:53.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.808+0000 7f0590ff9700 1 -- 172.21.15.7:0/2398465557 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4588 (secure 0 0 0) 0x7f058c009880 con 0x7f0594159690 2023-10-07T07:59:53.809 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:53.810 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3622385241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3622385241},{"type":"v1","addr":"172.21.15.172:6837","nonce":3622385241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/899764672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":899764672},{"type":"v1","addr":"172.21.15.172:6836","nonce":899764672}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/318366060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":318366060},{"type":"v1","addr":"172.21.15.7:6833","nonce":318366060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:59:49.797329+0000","modified":"2023-10-07T07:59:53.390983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/763397529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":763397529},{"type":"v1","addr":"172.21.15.7:6835","nonce":763397529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T07:59:53.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0578066f30 msgr2=0x7f05780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0578066f30 0x7f05780693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0588007fd0 tx=0x7f0588007d30 comp rx=0 tx=0).stop 2023-10-07T07:59:53.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594159690 msgr2=0x7f05940c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594159690 0x7f05940c6cd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0594069020 tx=0x7f058c019000 comp rx=0 tx=0).stop 2023-10-07T07:59:53.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 shutdown_connections 2023-10-07T07:59:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0578066f30 0x7f05780693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594159690 0x7f05940c6cd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05940cb0c0 0x7f05941554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 --2- 172.21.15.7:0/2398465557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05941592c0 0x7f05940ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 >> 172.21.15.7:0/2398465557 conn(0x7f0594068bd0 msgr2=0x7f059406a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 shutdown_connections 2023-10-07T07:59:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.810+0000 7f059ab4a700 1 -- 172.21.15.7:0/2398465557 wait complete. 2023-10-07T07:59:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2023-10-07T07:59:53.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T07:59:53.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.956+0000 7fc6a39be700 1 Processor -- start 2023-10-07T07:59:53.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.957+0000 7fc6a39be700 1 -- start start 2023-10-07T07:59:53.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.957+0000 7fc6a39be700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc69c159310 0x7fc69c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.957+0000 7fc6a39be700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69c068100 con 0x7fc69c1596e0 2023-10-07T07:59:53.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.957+0000 7fc6a39be700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69c068350 con 0x7fc69c0ced30 2023-10-07T07:59:53.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.957+0000 7fc6a39be700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c068630 con 0x7fc69c159310 2023-10-07T07:59:53.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.958+0000 7fc6a1f5b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc69c1596e0 0x7fc69c06b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34594/0 (socket says 172.21.15.7:34594) 2023-10-07T07:59:53.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.958+0000 7fc6a1f5b700 1 -- 172.21.15.7:0/1332441809 learned_addr learned my addr 172.21.15.7:0/1332441809 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:53.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.958+0000 7fc6a175a700 1 --2- 172.21.15.7:0/1332441809 >> v2:172.21.15.172:3300/0 conn(0x7fc69c159310 0x7fc69c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.958+0000 7fc693fff700 1 -- 172.21.15.7:0/1332441809 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674289073 0 0) 0x7fc69c068100 con 0x7fc69c1596e0 2023-10-07T07:59:53.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.958+0000 7fc693fff700 1 -- 172.21.15.7:0/1332441809 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc688003570 con 0x7fc69c1596e0 2023-10-07T07:59:53.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.958+0000 7fc6a175a700 1 -- 172.21.15.7:0/1332441809 >> v1:172.21.15.172:6789/0 conn(0x7fc69c0ced30 legacy=0x7fc69c0c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.958+0000 7fc6a175a700 1 -- 172.21.15.7:0/1332441809 >> v1:172.21.15.7:6789/0 conn(0x7fc69c1596e0 legacy=0x7fc69c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.958+0000 7fc6a175a700 1 -- 172.21.15.7:0/1332441809 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c068910 con 0x7fc69c159310 2023-10-07T07:59:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.958+0000 7fc6a175a700 1 --2- 172.21.15.7:0/1332441809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c159310 0x7fc69c0c86e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098e0 tx=0x7fc6980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0cd34d8fd5f03dc server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc693fff700 1 -- 172.21.15.7:0/1332441809 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800b640 con 0x7fc69c159310 2023-10-07T07:59:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc693fff700 1 -- 172.21.15.7:0/1332441809 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69800b7a0 con 0x7fc69c159310 2023-10-07T07:59:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc693fff700 1 -- 172.21.15.7:0/1332441809 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800bb40 con 0x7fc69c159310 2023-10-07T07:59:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc6a39be700 1 -- 172.21.15.7:0/1332441809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c159310 msgr2=0x7fc69c0c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc6a39be700 1 --2- 172.21.15.7:0/1332441809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c159310 0x7fc69c0c86e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098e0 tx=0x7fc6980091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc6a39be700 1 -- 172.21.15.7:0/1332441809 shutdown_connections 2023-10-07T07:59:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc6a39be700 1 --2- 172.21.15.7:0/1332441809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c159310 0x7fc69c0c86e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc6a39be700 1 -- 172.21.15.7:0/1332441809 >> 172.21.15.7:0/1332441809 conn(0x7fc69c067ac0 msgr2=0x7fc69c0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc6a39be700 1 -- 172.21.15.7:0/1332441809 shutdown_connections 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.959+0000 7fc6a39be700 1 -- 172.21.15.7:0/1332441809 wait complete. 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.960+0000 7fc6a39be700 1 Processor -- start 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.961+0000 7fc6a39be700 1 -- start start 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.961+0000 7fc6a39be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69c159310 0x7fc69c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.961+0000 7fc6a39be700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c1596e0 0x7fc69c16a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.961+0000 7fc6a39be700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c151e40 0x7fc69c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a39be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c152fa0 con 0x7fc69c159310 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a39be700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0b1f30 con 0x7fc69c151e40 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a39be700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c16abc0 con 0x7fc69c1596e0 2023-10-07T07:59:53.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a175a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69c159310 0x7fc69c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a175a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69c159310 0x7fc69c1533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38532/0 (socket says 172.21.15.7:38532) 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a175a700 1 -- 172.21.15.7:0/1899164278 learned_addr learned my addr 172.21.15.7:0/1899164278 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a1f5b700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c151e40 0x7fc69c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a0f59700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c1596e0 0x7fc69c16a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a175a700 1 -- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c1596e0 msgr2=0x7fc69c16a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a175a700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c1596e0 0x7fc69c16a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a175a700 1 -- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c151e40 msgr2=0x7fc69c156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a175a700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c151e40 0x7fc69c156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a175a700 1 -- 172.21.15.7:0/1899164278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698009420 con 0x7fc69c159310 2023-10-07T07:59:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a0f59700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c1596e0 0x7fc69c16a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a175a700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69c159310 0x7fc69c1533c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc698009f80 tx=0x7fc698017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6917fa700 1 -- 172.21.15.7:0/1899164278 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800bb20 con 0x7fc69c159310 2023-10-07T07:59:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6917fa700 1 -- 172.21.15.7:0/1899164278 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc698011420 con 0x7fc69c159310 2023-10-07T07:59:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c1573f0 con 0x7fc69c159310 2023-10-07T07:59:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6917fa700 1 -- 172.21.15.7:0/1899164278 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980117a0 con 0x7fc69c159310 2023-10-07T07:59:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.962+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c152600 con 0x7fc69c159310 2023-10-07T07:59:53.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.963+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c153210 con 0x7fc69c159310 2023-10-07T07:59:53.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.964+0000 7fc6917fa700 1 -- 172.21.15.7:0/1899164278 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc698011b10 con 0x7fc69c159310 2023-10-07T07:59:53.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.964+0000 7fc6917fa700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:53.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.964+0000 7fc6917fa700 1 -- 172.21.15.7:0/1899164278 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc698085cc0 con 0x7fc69c159310 2023-10-07T07:59:53.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.965+0000 7fc6a0f59700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:53.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.965+0000 7fc6a0f59700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc680066f30 0x7fc6800693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0b1b40 tx=0x7fc68c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:53.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:53.969+0000 7fc6917fa700 1 -- 172.21.15.7:0/1899164278 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc698052f30 con 0x7fc69c159310 2023-10-07T07:59:54.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.172+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc69c0c5080 con 0x7fc69c159310 2023-10-07T07:59:54.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.173+0000 7fc6917fa700 1 -- 172.21.15.7:0/1899164278 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fc698056550 con 0x7fc69c159310 2023-10-07T07:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":45,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T07:59:51.364609+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:59:48.327048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T07:59:48.760594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T07:59:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc680066f30 msgr2=0x7fc6800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc680066f30 0x7fc6800693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0b1b40 tx=0x7fc68c009040 comp rx=0 tx=0).stop 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69c159310 msgr2=0x7fc69c1533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69c159310 0x7fc69c1533c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc698009f80 tx=0x7fc698017000 comp rx=0 tx=0).stop 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 shutdown_connections 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc69c151e40 0x7fc69c156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69c159310 0x7fc69c1533c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 --2- 172.21.15.7:0/1899164278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc69c1596e0 0x7fc69c16a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 >> 172.21.15.7:0/1899164278 conn(0x7fc69c067ac0 msgr2=0x7fc69c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 shutdown_connections 2023-10-07T07:59:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.175+0000 7fc6a39be700 1 -- 172.21.15.7:0/1899164278 wait complete. 2023-10-07T07:59:54.190 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-07T07:59:54.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T07:59:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.321+0000 7f2fdcebb700 1 Processor -- start 2023-10-07T07:59:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.322+0000 7f2fdcebb700 1 -- start start 2023-10-07T07:59:54.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fdcebb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2fd80ced30 0x7f2fd80c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fdcebb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd8068100 con 0x7f2fd81596e0 2023-10-07T07:59:54.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fdcebb700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd8068350 con 0x7f2fd8159310 2023-10-07T07:59:54.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fdcebb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd8068630 con 0x7f2fd80ced30 2023-10-07T07:59:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fd6d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2fd81596e0 0x7f2fd806b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34608/0 (socket says 172.21.15.7:34608) 2023-10-07T07:59:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fd6d9d700 1 -- 172.21.15.7:0/2526425747 learned_addr learned my addr 172.21.15.7:0/2526425747 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fd5d9b700 1 --2- 172.21.15.7:0/2526425747 >> v2:172.21.15.172:3300/0 conn(0x7f2fd80ced30 0x7f2fd80c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fd559a700 1 -- 172.21.15.7:0/2526425747 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697343786 0 0) 0x7f2fd8068100 con 0x7f2fd81596e0 2023-10-07T07:59:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fd559a700 1 -- 172.21.15.7:0/2526425747 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc0003570 con 0x7f2fd81596e0 2023-10-07T07:59:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fd5d9b700 1 -- 172.21.15.7:0/2526425747 >> v1:172.21.15.172:6789/0 conn(0x7f2fd8159310 legacy=0x7f2fd80c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fd5d9b700 1 -- 172.21.15.7:0/2526425747 >> v1:172.21.15.7:6789/0 conn(0x7f2fd81596e0 legacy=0x7f2fd806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fd559a700 1 -- 172.21.15.7:0/2526425747 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3011339354 0 0) 0x7f2fc0003570 con 0x7f2fd81596e0 2023-10-07T07:59:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.323+0000 7f2fd5d9b700 1 -- 172.21.15.7:0/2526425747 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd8068910 con 0x7f2fd80ced30 2023-10-07T07:59:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fd5d9b700 1 --2- 172.21.15.7:0/2526425747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd80ced30 0x7f2fd80c86e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0098e0 tx=0x7f2fcc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=502649d242d96729 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fd559a700 1 -- 172.21.15.7:0/2526425747 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00017f0 con 0x7f2fd80ced30 2023-10-07T07:59:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fd559a700 1 -- 172.21.15.7:0/2526425747 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc011420 con 0x7f2fd80ced30 2023-10-07T07:59:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fd559a700 1 -- 172.21.15.7:0/2526425747 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0117c0 con 0x7f2fd80ced30 2023-10-07T07:59:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fdcebb700 1 -- 172.21.15.7:0/2526425747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd80ced30 msgr2=0x7f2fd80c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fdcebb700 1 --2- 172.21.15.7:0/2526425747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd80ced30 0x7f2fd80c86e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0098e0 tx=0x7f2fcc0091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fdcebb700 1 -- 172.21.15.7:0/2526425747 shutdown_connections 2023-10-07T07:59:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fdcebb700 1 --2- 172.21.15.7:0/2526425747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd80ced30 0x7f2fd80c86e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fdcebb700 1 -- 172.21.15.7:0/2526425747 >> 172.21.15.7:0/2526425747 conn(0x7f2fd8067ac0 msgr2=0x7f2fd80ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fdcebb700 1 -- 172.21.15.7:0/2526425747 shutdown_connections 2023-10-07T07:59:54.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.324+0000 7f2fdcebb700 1 -- 172.21.15.7:0/2526425747 wait complete. 2023-10-07T07:59:54.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.326+0000 7f2fdcebb700 1 Processor -- start 2023-10-07T07:59:54.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.326+0000 7f2fdcebb700 1 -- start start 2023-10-07T07:59:54.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.326+0000 7f2fdcebb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fd8159310 0x7f2fd80cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.326+0000 7f2fdcebb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd81596e0 0x7f2fd81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.326+0000 7f2fdcebb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd815c320 0x7f2fd816d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.326+0000 7f2fdcebb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd80b1c60 con 0x7f2fd815c320 2023-10-07T07:59:54.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.326+0000 7f2fdcebb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd815a6e0 con 0x7f2fd81596e0 2023-10-07T07:59:54.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.326+0000 7f2fdcebb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd815a980 con 0x7f2fd8159310 2023-10-07T07:59:54.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd815c320 0x7f2fd816d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd815c320 0x7f2fd816d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38540/0 (socket says 172.21.15.7:38540) 2023-10-07T07:59:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd6d9d700 1 -- 172.21.15.7:0/589538481 learned_addr learned my addr 172.21.15.7:0/589538481 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd5d9b700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd81596e0 0x7f2fd81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd659c700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fd8159310 0x7f2fd80cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd6d9d700 1 -- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fd8159310 msgr2=0x7f2fd80cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd6d9d700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fd8159310 0x7f2fd80cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd6d9d700 1 -- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd81596e0 msgr2=0x7f2fd81531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd6d9d700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd81596e0 0x7f2fd81531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd6d9d700 1 -- 172.21.15.7:0/589538481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc009420 con 0x7f2fd815c320 2023-10-07T07:59:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd5d9b700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd81596e0 0x7f2fd81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fd6d9d700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd815c320 0x7f2fd816d0b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000cb90 tx=0x7f2fd0007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fc67fc700 1 -- 172.21.15.7:0/589538481 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0004dc0 con 0x7f2fd815c320 2023-10-07T07:59:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fc67fc700 1 -- 172.21.15.7:0/589538481 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd000a700 con 0x7f2fd815c320 2023-10-07T07:59:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd815ac60 con 0x7f2fd815c320 2023-10-07T07:59:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fc67fc700 1 -- 172.21.15.7:0/589538481 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000aac0 con 0x7f2fd815c320 2023-10-07T07:59:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.327+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd80b2190 con 0x7f2fd815c320 2023-10-07T07:59:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.328+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd806ddf0 con 0x7f2fd815c320 2023-10-07T07:59:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.329+0000 7f2fc67fc700 1 -- 172.21.15.7:0/589538481 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fd0007150 con 0x7f2fd815c320 2023-10-07T07:59:54.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.329+0000 7f2fc67fc700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fb8066f30 0x7f2fb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.329+0000 7f2fc67fc700 1 -- 172.21.15.7:0/589538481 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fd0084100 con 0x7f2fd815c320 2023-10-07T07:59:54.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.329+0000 7f2fd659c700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fb8066f30 0x7f2fb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.330+0000 7f2fd659c700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fb8066f30 0x7f2fb80693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8003860 tx=0x7f2fc8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:54.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.334+0000 7f2fc67fc700 1 -- 172.21.15.7:0/589538481 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fd00513f0 con 0x7f2fd815c320 2023-10-07T07:59:54.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.532+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2fd80b2380 con 0x7f2fb8066f30 2023-10-07T07:59:54.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.541+0000 7f2fc67fc700 1 -- 172.21.15.7:0/589538481 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289526 (secure 0 0 0) 0x7f2fd80b2380 con 0x7f2fb8066f30 2023-10-07T07:59:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fb8066f30 msgr2=0x7f2fb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fb8066f30 0x7f2fb80693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8003860 tx=0x7f2fc8009040 comp rx=0 tx=0).stop 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd815c320 msgr2=0x7f2fd816d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd815c320 0x7f2fd816d0b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000cb90 tx=0x7f2fd0007ca0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 shutdown_connections 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2fb8066f30 0x7f2fb80693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd815c320 0x7f2fd816d0b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fd8159310 0x7f2fd80cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 --2- 172.21.15.7:0/589538481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fd81596e0 0x7f2fd81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 >> 172.21.15.7:0/589538481 conn(0x7f2fd8067ac0 msgr2=0x7f2fd806d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 shutdown_connections 2023-10-07T07:59:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.545+0000 7f2fdcebb700 1 -- 172.21.15.7:0/589538481 wait complete. 2023-10-07T07:59:54.549 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T07:59:54.560 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-10-07T07:59:54.055376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383656,"kb_used_data":5352,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591064,"statfs":{"total":773094113280,"available":770653249536,"internally_reserved":0,"allocated":5480448,"data_stored":2891612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.702572"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.392695+0000","last_change":"2023-10-07T07:59:50.391481+0000","last_active":"2023-10-07T07:59:50.392695+0000","last_peered":"2023-10-07T07:59:50.392695+0000","last_clean":"2023-10-07T07:59:50.392695+0000","last_became_active":"2023-10-07T07:59:50.390948+0000","last_became_peered":"2023-10-07T07:59:50.390948+0000","last_unstale":"2023-10-07T07:59:50.392695+0000","last_undegraded":"2023-10-07T07:59:50.392695+0000","last_fullsized":"2023-10-07T07:59:50.392695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:24:48.018565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393362+0000","last_change":"2023-10-07T07:59:50.389001+0000","last_active":"2023-10-07T07:59:50.393362+0000","last_peered":"2023-10-07T07:59:50.393362+0000","last_clean":"2023-10-07T07:59:50.393362+0000","last_became_active":"2023-10-07T07:59:50.385542+0000","last_became_peered":"2023-10-07T07:59:50.385542+0000","last_unstale":"2023-10-07T07:59:50.393362+0000","last_undegraded":"2023-10-07T07:59:50.393362+0000","last_fullsized":"2023-10-07T07:59:50.393362+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:24.742162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.389006+0000","last_change":"2023-10-07T07:59:50.383670+0000","last_active":"2023-10-07T07:59:50.389006+0000","last_peered":"2023-10-07T07:59:50.389006+0000","last_clean":"2023-10-07T07:59:50.389006+0000","last_became_active":"2023-10-07T07:59:50.383343+0000","last_became_peered":"2023-10-07T07:59:50.383343+0000","last_unstale":"2023-10-07T07:59:50.389006+0000","last_undegraded":"2023-10-07T07:59:50.389006+0000","last_fullsized":"2023-10-07T07:59:50.389006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:17:52.050217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382762+0000","last_change":"2023-10-07T07:59:50.374608+0000","last_active":"2023-10-07T07:59:50.382762+0000","last_peered":"2023-10-07T07:59:50.382762+0000","last_clean":"2023-10-07T07:59:50.382762+0000","last_became_active":"2023-10-07T07:59:50.374347+0000","last_became_peered":"2023-10-07T07:59:50.374347+0000","last_unstale":"2023-10-07T07:59:50.382762+0000","last_undegraded":"2023-10-07T07:59:50.382762+0000","last_fullsized":"2023-10-07T07:59:50.382762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:19:08.600628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382530+0000","last_change":"2023-10-07T07:59:50.379141+0000","last_active":"2023-10-07T07:59:50.382530+0000","last_peered":"2023-10-07T07:59:50.382530+0000","last_clean":"2023-10-07T07:59:50.382530+0000","last_became_active":"2023-10-07T07:59:50.378816+0000","last_became_peered":"2023-10-07T07:59:50.378816+0000","last_unstale":"2023-10-07T07:59:50.382530+0000","last_undegraded":"2023-10-07T07:59:50.382530+0000","last_fullsized":"2023-10-07T07:59:50.382530+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:27:58.785440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.377924+0000","last_change":"2023-10-07T07:59:50.370620+0000","last_active":"2023-10-07T07:59:50.377924+0000","last_peered":"2023-10-07T07:59:50.377924+0000","last_clean":"2023-10-07T07:59:50.377924+0000","last_became_active":"2023-10-07T07:59:50.370261+0000","last_became_peered":"2023-10-07T07:59:50.370261+0000","last_unstale":"2023-10-07T07:59:50.377924+0000","last_undegraded":"2023-10-07T07:59:50.377924+0000","last_fullsized":"2023-10-07T07:59:50.377924+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:24.742162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388206+0000","last_change":"2023-10-07T07:59:50.379891+0000","last_active":"2023-10-07T07:59:50.388206+0000","last_peered":"2023-10-07T07:59:50.388206+0000","last_clean":"2023-10-07T07:59:50.388206+0000","last_became_active":"2023-10-07T07:59:50.378815+0000","last_became_peered":"2023-10-07T07:59:50.378815+0000","last_unstale":"2023-10-07T07:59:50.388206+0000","last_undegraded":"2023-10-07T07:59:50.388206+0000","last_fullsized":"2023-10-07T07:59:50.388206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:33.961667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.385352+0000","last_change":"2023-10-07T07:59:50.380131+0000","last_active":"2023-10-07T07:59:50.385352+0000","last_peered":"2023-10-07T07:59:50.385352+0000","last_clean":"2023-10-07T07:59:50.385352+0000","last_became_active":"2023-10-07T07:59:50.379887+0000","last_became_peered":"2023-10-07T07:59:50.379887+0000","last_unstale":"2023-10-07T07:59:50.385352+0000","last_undegraded":"2023-10-07T07:59:50.385352+0000","last_fullsized":"2023-10-07T07:59:50.385352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:40:48.155422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.385351+0000","last_change":"2023-10-07T07:59:50.380389+0000","last_active":"2023-10-07T07:59:50.385351+0000","last_peered":"2023-10-07T07:59:50.385351+0000","last_clean":"2023-10-07T07:59:50.385351+0000","last_became_active":"2023-10-07T07:59:50.380091+0000","last_became_peered":"2023-10-07T07:59:50.380091+0000","last_unstale":"2023-10-07T07:59:50.385351+0000","last_undegraded":"2023-10-07T07:59:50.385351+0000","last_fullsized":"2023-10-07T07:59:50.385351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:13.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394204+0000","last_change":"2023-10-07T07:59:50.390182+0000","last_active":"2023-10-07T07:59:50.394204+0000","last_peered":"2023-10-07T07:59:50.394204+0000","last_clean":"2023-10-07T07:59:50.394204+0000","last_became_active":"2023-10-07T07:59:50.389924+0000","last_became_peered":"2023-10-07T07:59:50.389924+0000","last_unstale":"2023-10-07T07:59:50.394204+0000","last_undegraded":"2023-10-07T07:59:50.394204+0000","last_fullsized":"2023-10-07T07:59:50.394204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:01:40.690852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.378170+0000","last_change":"2023-10-07T07:59:50.373976+0000","last_active":"2023-10-07T07:59:50.378170+0000","last_peered":"2023-10-07T07:59:50.378170+0000","last_clean":"2023-10-07T07:59:50.378170+0000","last_became_active":"2023-10-07T07:59:50.370357+0000","last_became_peered":"2023-10-07T07:59:50.370357+0000","last_unstale":"2023-10-07T07:59:50.378170+0000","last_undegraded":"2023-10-07T07:59:50.378170+0000","last_fullsized":"2023-10-07T07:59:50.378170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:02:53.923228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388901+0000","last_change":"2023-10-07T07:59:50.375292+0000","last_active":"2023-10-07T07:59:50.388901+0000","last_peered":"2023-10-07T07:59:50.388901+0000","last_clean":"2023-10-07T07:59:50.388901+0000","last_became_active":"2023-10-07T07:59:50.374983+0000","last_became_peered":"2023-10-07T07:59:50.374983+0000","last_unstale":"2023-10-07T07:59:50.388901+0000","last_undegraded":"2023-10-07T07:59:50.388901+0000","last_fullsized":"2023-10-07T07:59:50.388901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:40:45.957885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.383246+0000","last_change":"2023-10-07T07:59:50.375624+0000","last_active":"2023-10-07T07:59:50.383246+0000","last_peered":"2023-10-07T07:59:50.383246+0000","last_clean":"2023-10-07T07:59:50.383246+0000","last_became_active":"2023-10-07T07:59:50.375350+0000","last_became_peered":"2023-10-07T07:59:50.375350+0000","last_unstale":"2023-10-07T07:59:50.383246+0000","last_undegraded":"2023-10-07T07:59:50.383246+0000","last_fullsized":"2023-10-07T07:59:50.383246+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:21:00.581364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384299+0000","last_change":"2023-10-07T07:59:50.376296+0000","last_active":"2023-10-07T07:59:50.384299+0000","last_peered":"2023-10-07T07:59:50.384299+0000","last_clean":"2023-10-07T07:59:50.384299+0000","last_became_active":"2023-10-07T07:59:50.376014+0000","last_became_peered":"2023-10-07T07:59:50.376014+0000","last_unstale":"2023-10-07T07:59:50.384299+0000","last_undegraded":"2023-10-07T07:59:50.384299+0000","last_fullsized":"2023-10-07T07:59:50.384299+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:07:24.396636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384541+0000","last_change":"2023-10-07T07:59:50.378694+0000","last_active":"2023-10-07T07:59:50.384541+0000","last_peered":"2023-10-07T07:59:50.384541+0000","last_clean":"2023-10-07T07:59:50.384541+0000","last_became_active":"2023-10-07T07:59:50.378277+0000","last_became_peered":"2023-10-07T07:59:50.378277+0000","last_unstale":"2023-10-07T07:59:50.384541+0000","last_undegraded":"2023-10-07T07:59:50.384541+0000","last_fullsized":"2023-10-07T07:59:50.384541+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:37:30.025946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.383983+0000","last_change":"2023-10-07T07:59:50.380265+0000","last_active":"2023-10-07T07:59:50.383983+0000","last_peered":"2023-10-07T07:59:50.383983+0000","last_clean":"2023-10-07T07:59:50.383983+0000","last_became_active":"2023-10-07T07:59:50.379966+0000","last_became_peered":"2023-10-07T07:59:50.379966+0000","last_unstale":"2023-10-07T07:59:50.383983+0000","last_undegraded":"2023-10-07T07:59:50.383983+0000","last_fullsized":"2023-10-07T07:59:50.383983+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:04:07.523857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394017+0000","last_change":"2023-10-07T07:59:50.389873+0000","last_active":"2023-10-07T07:59:50.394017+0000","last_peered":"2023-10-07T07:59:50.394017+0000","last_clean":"2023-10-07T07:59:50.394017+0000","last_became_active":"2023-10-07T07:59:50.388544+0000","last_became_peered":"2023-10-07T07:59:50.388544+0000","last_unstale":"2023-10-07T07:59:50.394017+0000","last_undegraded":"2023-10-07T07:59:50.394017+0000","last_fullsized":"2023-10-07T07:59:50.394017+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:13.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.386882+0000","last_change":"2023-10-07T07:59:50.377558+0000","last_active":"2023-10-07T07:59:50.386882+0000","last_peered":"2023-10-07T07:59:50.386882+0000","last_clean":"2023-10-07T07:59:50.386882+0000","last_became_active":"2023-10-07T07:59:50.377246+0000","last_became_peered":"2023-10-07T07:59:50.377246+0000","last_unstale":"2023-10-07T07:59:50.386882+0000","last_undegraded":"2023-10-07T07:59:50.386882+0000","last_fullsized":"2023-10-07T07:59:50.386882+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:01:02.584241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384898+0000","last_change":"2023-10-07T07:59:50.381205+0000","last_active":"2023-10-07T07:59:50.384898+0000","last_peered":"2023-10-07T07:59:50.384898+0000","last_clean":"2023-10-07T07:59:50.384898+0000","last_became_active":"2023-10-07T07:59:50.380889+0000","last_became_peered":"2023-10-07T07:59:50.380889+0000","last_unstale":"2023-10-07T07:59:50.384898+0000","last_undegraded":"2023-10-07T07:59:50.384898+0000","last_fullsized":"2023-10-07T07:59:50.384898+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:14:18.711278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390538+0000","last_change":"2023-10-07T07:59:50.383548+0000","last_active":"2023-10-07T07:59:50.390538+0000","last_peered":"2023-10-07T07:59:50.390538+0000","last_clean":"2023-10-07T07:59:50.390538+0000","last_became_active":"2023-10-07T07:59:50.383165+0000","last_became_peered":"2023-10-07T07:59:50.383165+0000","last_unstale":"2023-10-07T07:59:50.390538+0000","last_undegraded":"2023-10-07T07:59:50.390538+0000","last_fullsized":"2023-10-07T07:59:50.390538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:37:30.025946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384502+0000","last_change":"2023-10-07T07:59:50.376768+0000","last_active":"2023-10-07T07:59:50.384502+0000","last_peered":"2023-10-07T07:59:50.384502+0000","last_clean":"2023-10-07T07:59:50.384502+0000","last_became_active":"2023-10-07T07:59:50.376432+0000","last_became_peered":"2023-10-07T07:59:50.376432+0000","last_unstale":"2023-10-07T07:59:50.384502+0000","last_undegraded":"2023-10-07T07:59:50.384502+0000","last_fullsized":"2023-10-07T07:59:50.384502+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:19:58.637016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.378282+0000","last_change":"2023-10-07T07:59:50.370978+0000","last_active":"2023-10-07T07:59:50.378282+0000","last_peered":"2023-10-07T07:59:50.378282+0000","last_clean":"2023-10-07T07:59:50.378282+0000","last_became_active":"2023-10-07T07:59:50.370712+0000","last_became_peered":"2023-10-07T07:59:50.370712+0000","last_unstale":"2023-10-07T07:59:50.378282+0000","last_undegraded":"2023-10-07T07:59:50.378282+0000","last_fullsized":"2023-10-07T07:59:50.378282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:33:35.420235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.389619+0000","last_change":"2023-10-07T07:59:50.383265+0000","last_active":"2023-10-07T07:59:50.389619+0000","last_peered":"2023-10-07T07:59:50.389619+0000","last_clean":"2023-10-07T07:59:50.389619+0000","last_became_active":"2023-10-07T07:59:50.381574+0000","last_became_peered":"2023-10-07T07:59:50.381574+0000","last_unstale":"2023-10-07T07:59:50.389619+0000","last_undegraded":"2023-10-07T07:59:50.389619+0000","last_fullsized":"2023-10-07T07:59:50.389619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:44:26.921725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393424+0000","last_change":"2023-10-07T07:59:50.387676+0000","last_active":"2023-10-07T07:59:50.393424+0000","last_peered":"2023-10-07T07:59:50.393424+0000","last_clean":"2023-10-07T07:59:50.393424+0000","last_became_active":"2023-10-07T07:59:50.378692+0000","last_became_peered":"2023-10-07T07:59:50.378692+0000","last_unstale":"2023-10-07T07:59:50.393424+0000","last_undegraded":"2023-10-07T07:59:50.393424+0000","last_fullsized":"2023-10-07T07:59:50.393424+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:37:28.888391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.392671+0000","last_change":"2023-10-07T07:59:50.391186+0000","last_active":"2023-10-07T07:59:50.392671+0000","last_peered":"2023-10-07T07:59:50.392671+0000","last_clean":"2023-10-07T07:59:50.392671+0000","last_became_active":"2023-10-07T07:59:50.390585+0000","last_became_peered":"2023-10-07T07:59:50.390585+0000","last_unstale":"2023-10-07T07:59:50.392671+0000","last_undegraded":"2023-10-07T07:59:50.392671+0000","last_fullsized":"2023-10-07T07:59:50.392671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:37:28.888391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.378902+0000","last_change":"2023-10-07T07:59:50.371153+0000","last_active":"2023-10-07T07:59:50.378902+0000","last_peered":"2023-10-07T07:59:50.378902+0000","last_clean":"2023-10-07T07:59:50.378902+0000","last_became_active":"2023-10-07T07:59:50.370893+0000","last_became_peered":"2023-10-07T07:59:50.370893+0000","last_unstale":"2023-10-07T07:59:50.378902+0000","last_undegraded":"2023-10-07T07:59:50.378902+0000","last_fullsized":"2023-10-07T07:59:50.378902+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:22:45.457050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.395633+0000","last_change":"2023-10-07T07:59:50.392978+0000","last_active":"2023-10-07T07:59:50.395633+0000","last_peered":"2023-10-07T07:59:50.395633+0000","last_clean":"2023-10-07T07:59:50.395633+0000","last_became_active":"2023-10-07T07:59:50.392660+0000","last_became_peered":"2023-10-07T07:59:50.392660+0000","last_unstale":"2023-10-07T07:59:50.395633+0000","last_undegraded":"2023-10-07T07:59:50.395633+0000","last_fullsized":"2023-10-07T07:59:50.395633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:25:19.271682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382961+0000","last_change":"2023-10-07T07:59:50.376540+0000","last_active":"2023-10-07T07:59:50.382961+0000","last_peered":"2023-10-07T07:59:50.382961+0000","last_clean":"2023-10-07T07:59:50.382961+0000","last_became_active":"2023-10-07T07:59:50.375928+0000","last_became_peered":"2023-10-07T07:59:50.375928+0000","last_unstale":"2023-10-07T07:59:50.382961+0000","last_undegraded":"2023-10-07T07:59:50.382961+0000","last_fullsized":"2023-10-07T07:59:50.382961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:16:52.133679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.386779+0000","last_change":"2023-10-07T07:59:50.385727+0000","last_active":"2023-10-07T07:59:50.386779+0000","last_peered":"2023-10-07T07:59:50.386779+0000","last_clean":"2023-10-07T07:59:50.386779+0000","last_became_active":"2023-10-07T07:59:50.385469+0000","last_became_peered":"2023-10-07T07:59:50.385469+0000","last_unstale":"2023-10-07T07:59:50.386779+0000","last_undegraded":"2023-10-07T07:59:50.386779+0000","last_fullsized":"2023-10-07T07:59:50.386779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:43:44.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393628+0000","last_change":"2023-10-07T07:59:50.389947+0000","last_active":"2023-10-07T07:59:50.393628+0000","last_peered":"2023-10-07T07:59:50.393628+0000","last_clean":"2023-10-07T07:59:50.393628+0000","last_became_active":"2023-10-07T07:59:50.388636+0000","last_became_peered":"2023-10-07T07:59:50.388636+0000","last_unstale":"2023-10-07T07:59:50.393628+0000","last_undegraded":"2023-10-07T07:59:50.393628+0000","last_fullsized":"2023-10-07T07:59:50.393628+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:24:48.018565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384353+0000","last_change":"2023-10-07T07:59:50.379291+0000","last_active":"2023-10-07T07:59:50.384353+0000","last_peered":"2023-10-07T07:59:50.384353+0000","last_clean":"2023-10-07T07:59:50.384353+0000","last_became_active":"2023-10-07T07:59:50.378875+0000","last_became_peered":"2023-10-07T07:59:50.378875+0000","last_unstale":"2023-10-07T07:59:50.384353+0000","last_undegraded":"2023-10-07T07:59:50.384353+0000","last_fullsized":"2023-10-07T07:59:50.384353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:43:47.947126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.376488+0000","last_change":"2023-10-07T07:59:50.369149+0000","last_active":"2023-10-07T07:59:50.376488+0000","last_peered":"2023-10-07T07:59:50.376488+0000","last_clean":"2023-10-07T07:59:50.376488+0000","last_became_active":"2023-10-07T07:59:50.368914+0000","last_became_peered":"2023-10-07T07:59:50.368914+0000","last_unstale":"2023-10-07T07:59:50.376488+0000","last_undegraded":"2023-10-07T07:59:50.376488+0000","last_fullsized":"2023-10-07T07:59:50.376488+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:40:41.701801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382454+0000","last_change":"2023-10-07T07:59:50.377568+0000","last_active":"2023-10-07T07:59:50.382454+0000","last_peered":"2023-10-07T07:59:50.382454+0000","last_clean":"2023-10-07T07:59:50.382454+0000","last_became_active":"2023-10-07T07:59:50.377139+0000","last_became_peered":"2023-10-07T07:59:50.377139+0000","last_unstale":"2023-10-07T07:59:50.382454+0000","last_undegraded":"2023-10-07T07:59:50.382454+0000","last_fullsized":"2023-10-07T07:59:50.382454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:31:21.880962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.386265+0000","last_change":"2023-10-07T07:59:49.371877+0000","last_active":"2023-10-07T07:59:50.386265+0000","last_peered":"2023-10-07T07:59:50.386265+0000","last_clean":"2023-10-07T07:59:50.386265+0000","last_became_active":"2023-10-07T07:59:49.371606+0000","last_became_peered":"2023-10-07T07:59:49.371606+0000","last_unstale":"2023-10-07T07:59:50.386265+0000","last_undegraded":"2023-10-07T07:59:50.386265+0000","last_fullsized":"2023-10-07T07:59:50.386265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:29.500217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.386217+0000","last_change":"2023-10-07T07:59:50.384602+0000","last_active":"2023-10-07T07:59:50.386217+0000","last_peered":"2023-10-07T07:59:50.386217+0000","last_clean":"2023-10-07T07:59:50.386217+0000","last_became_active":"2023-10-07T07:59:50.384255+0000","last_became_peered":"2023-10-07T07:59:50.384255+0000","last_unstale":"2023-10-07T07:59:50.386217+0000","last_undegraded":"2023-10-07T07:59:50.386217+0000","last_fullsized":"2023-10-07T07:59:50.386217+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:19:39.418430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393745+0000","last_change":"2023-10-07T07:59:49.380830+0000","last_active":"2023-10-07T07:59:50.393745+0000","last_peered":"2023-10-07T07:59:50.393745+0000","last_clean":"2023-10-07T07:59:50.393745+0000","last_became_active":"2023-10-07T07:59:49.380454+0000","last_became_peered":"2023-10-07T07:59:49.380454+0000","last_unstale":"2023-10-07T07:59:50.393745+0000","last_undegraded":"2023-10-07T07:59:50.393745+0000","last_fullsized":"2023-10-07T07:59:50.393745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:22:44.452205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393666+0000","last_change":"2023-10-07T07:59:50.389770+0000","last_active":"2023-10-07T07:59:50.393666+0000","last_peered":"2023-10-07T07:59:50.393666+0000","last_clean":"2023-10-07T07:59:50.393666+0000","last_became_active":"2023-10-07T07:59:50.385593+0000","last_became_peered":"2023-10-07T07:59:50.385593+0000","last_unstale":"2023-10-07T07:59:50.393666+0000","last_undegraded":"2023-10-07T07:59:50.393666+0000","last_fullsized":"2023-10-07T07:59:50.393666+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:45:09.763991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.385656+0000","last_change":"2023-10-07T07:59:49.373004+0000","last_active":"2023-10-07T07:59:50.385656+0000","last_peered":"2023-10-07T07:59:50.385656+0000","last_clean":"2023-10-07T07:59:50.385656+0000","last_became_active":"2023-10-07T07:59:49.371664+0000","last_became_peered":"2023-10-07T07:59:49.371664+0000","last_unstale":"2023-10-07T07:59:50.385656+0000","last_undegraded":"2023-10-07T07:59:50.385656+0000","last_fullsized":"2023-10-07T07:59:50.385656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:45:03.778103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384332+0000","last_change":"2023-10-07T07:59:50.375337+0000","last_active":"2023-10-07T07:59:50.384332+0000","last_peered":"2023-10-07T07:59:50.384332+0000","last_clean":"2023-10-07T07:59:50.384332+0000","last_became_active":"2023-10-07T07:59:50.375090+0000","last_became_peered":"2023-10-07T07:59:50.375090+0000","last_unstale":"2023-10-07T07:59:50.384332+0000","last_undegraded":"2023-10-07T07:59:50.384332+0000","last_fullsized":"2023-10-07T07:59:50.384332+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:52:49.742912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394274+0000","last_change":"2023-10-07T07:59:49.380751+0000","last_active":"2023-10-07T07:59:50.394274+0000","last_peered":"2023-10-07T07:59:50.394274+0000","last_clean":"2023-10-07T07:59:50.394274+0000","last_became_active":"2023-10-07T07:59:49.380422+0000","last_became_peered":"2023-10-07T07:59:49.380422+0000","last_unstale":"2023-10-07T07:59:50.394274+0000","last_undegraded":"2023-10-07T07:59:50.394274+0000","last_fullsized":"2023-10-07T07:59:50.394274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:02:52.918383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.392476+0000","last_change":"2023-10-07T07:59:50.383246+0000","last_active":"2023-10-07T07:59:50.392476+0000","last_peered":"2023-10-07T07:59:50.392476+0000","last_clean":"2023-10-07T07:59:50.392476+0000","last_became_active":"2023-10-07T07:59:50.382897+0000","last_became_peered":"2023-10-07T07:59:50.382897+0000","last_unstale":"2023-10-07T07:59:50.392476+0000","last_undegraded":"2023-10-07T07:59:50.392476+0000","last_fullsized":"2023-10-07T07:59:50.392476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:33:16.341323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.395405+0000","last_change":"2023-10-07T07:59:49.373658+0000","last_active":"2023-10-07T07:59:50.395405+0000","last_peered":"2023-10-07T07:59:50.395405+0000","last_clean":"2023-10-07T07:59:50.395405+0000","last_became_active":"2023-10-07T07:59:49.373384+0000","last_became_peered":"2023-10-07T07:59:49.373384+0000","last_unstale":"2023-10-07T07:59:50.395405+0000","last_undegraded":"2023-10-07T07:59:50.395405+0000","last_fullsized":"2023-10-07T07:59:50.395405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:31:20.876117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.389932+0000","last_change":"2023-10-07T07:59:50.379482+0000","last_active":"2023-10-07T07:59:50.389932+0000","last_peered":"2023-10-07T07:59:50.389932+0000","last_clean":"2023-10-07T07:59:50.389932+0000","last_became_active":"2023-10-07T07:59:50.379174+0000","last_became_peered":"2023-10-07T07:59:50.379174+0000","last_unstale":"2023-10-07T07:59:50.389932+0000","last_undegraded":"2023-10-07T07:59:50.389932+0000","last_fullsized":"2023-10-07T07:59:50.389932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:43.243398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.392949+0000","last_change":"2023-10-07T07:59:49.378611+0000","last_active":"2023-10-07T07:59:50.392949+0000","last_peered":"2023-10-07T07:59:50.392949+0000","last_clean":"2023-10-07T07:59:50.392949+0000","last_became_active":"2023-10-07T07:59:49.377794+0000","last_became_peered":"2023-10-07T07:59:49.377794+0000","last_unstale":"2023-10-07T07:59:50.392949+0000","last_undegraded":"2023-10-07T07:59:50.392949+0000","last_fullsized":"2023-10-07T07:59:50.392949+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:25:30.259592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390450+0000","last_change":"2023-10-07T07:59:50.381157+0000","last_active":"2023-10-07T07:59:50.390450+0000","last_peered":"2023-10-07T07:59:50.390450+0000","last_clean":"2023-10-07T07:59:50.390450+0000","last_became_active":"2023-10-07T07:59:50.380898+0000","last_became_peered":"2023-10-07T07:59:50.380898+0000","last_unstale":"2023-10-07T07:59:50.390450+0000","last_undegraded":"2023-10-07T07:59:50.390450+0000","last_fullsized":"2023-10-07T07:59:50.390450+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:07:24.396636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.376037+0000","last_change":"2023-10-07T07:59:49.376511+0000","last_active":"2023-10-07T07:59:50.376037+0000","last_peered":"2023-10-07T07:59:50.376037+0000","last_clean":"2023-10-07T07:59:50.376037+0000","last_became_active":"2023-10-07T07:59:49.376207+0000","last_became_peered":"2023-10-07T07:59:49.376207+0000","last_unstale":"2023-10-07T07:59:50.376037+0000","last_undegraded":"2023-10-07T07:59:50.376037+0000","last_fullsized":"2023-10-07T07:59:50.376037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:12:16.779766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388234+0000","last_change":"2023-10-07T07:59:50.379115+0000","last_active":"2023-10-07T07:59:50.388234+0000","last_peered":"2023-10-07T07:59:50.388234+0000","last_clean":"2023-10-07T07:59:50.388234+0000","last_became_active":"2023-10-07T07:59:50.378371+0000","last_became_peered":"2023-10-07T07:59:50.378371+0000","last_unstale":"2023-10-07T07:59:50.388234+0000","last_undegraded":"2023-10-07T07:59:50.388234+0000","last_fullsized":"2023-10-07T07:59:50.388234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:25:09.893988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.373624+0000","last_change":"2023-10-07T07:59:49.376140+0000","last_active":"2023-10-07T07:59:50.373624+0000","last_peered":"2023-10-07T07:59:50.373624+0000","last_clean":"2023-10-07T07:59:50.373624+0000","last_became_active":"2023-10-07T07:59:49.375871+0000","last_became_peered":"2023-10-07T07:59:49.375871+0000","last_unstale":"2023-10-07T07:59:50.373624+0000","last_undegraded":"2023-10-07T07:59:50.373624+0000","last_fullsized":"2023-10-07T07:59:50.373624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:24:47.013720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388768+0000","last_change":"2023-10-07T07:59:50.384640+0000","last_active":"2023-10-07T07:59:50.388768+0000","last_peered":"2023-10-07T07:59:50.388768+0000","last_clean":"2023-10-07T07:59:50.388768+0000","last_became_active":"2023-10-07T07:59:50.384336+0000","last_became_peered":"2023-10-07T07:59:50.384336+0000","last_unstale":"2023-10-07T07:59:50.388768+0000","last_undegraded":"2023-10-07T07:59:50.388768+0000","last_fullsized":"2023-10-07T07:59:50.388768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:33:35.420235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.371756+0000","last_change":"2023-10-07T07:59:49.375115+0000","last_active":"2023-10-07T07:59:50.371756+0000","last_peered":"2023-10-07T07:59:50.371756+0000","last_clean":"2023-10-07T07:59:50.371756+0000","last_became_active":"2023-10-07T07:59:49.374846+0000","last_became_peered":"2023-10-07T07:59:49.374846+0000","last_unstale":"2023-10-07T07:59:50.371756+0000","last_undegraded":"2023-10-07T07:59:50.371756+0000","last_fullsized":"2023-10-07T07:59:50.371756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:59:08.038668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388928+0000","last_change":"2023-10-07T07:59:50.384852+0000","last_active":"2023-10-07T07:59:50.388928+0000","last_peered":"2023-10-07T07:59:50.388928+0000","last_clean":"2023-10-07T07:59:50.388928+0000","last_became_active":"2023-10-07T07:59:50.384559+0000","last_became_peered":"2023-10-07T07:59:50.384559+0000","last_unstale":"2023-10-07T07:59:50.388928+0000","last_undegraded":"2023-10-07T07:59:50.388928+0000","last_fullsized":"2023-10-07T07:59:50.388928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:56:18.818754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.374777+0000","last_change":"2023-10-07T07:59:49.375370+0000","last_active":"2023-10-07T07:59:50.374777+0000","last_peered":"2023-10-07T07:59:50.374777+0000","last_clean":"2023-10-07T07:59:50.374777+0000","last_became_active":"2023-10-07T07:59:49.374821+0000","last_became_peered":"2023-10-07T07:59:49.374821+0000","last_unstale":"2023-10-07T07:59:50.374777+0000","last_undegraded":"2023-10-07T07:59:50.374777+0000","last_fullsized":"2023-10-07T07:59:50.374777+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:44:23.414684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.385323+0000","last_change":"2023-10-07T07:59:50.379896+0000","last_active":"2023-10-07T07:59:50.385323+0000","last_peered":"2023-10-07T07:59:50.385323+0000","last_clean":"2023-10-07T07:59:50.385323+0000","last_became_active":"2023-10-07T07:59:50.379598+0000","last_became_peered":"2023-10-07T07:59:50.379598+0000","last_unstale":"2023-10-07T07:59:50.385323+0000","last_undegraded":"2023-10-07T07:59:50.385323+0000","last_fullsized":"2023-10-07T07:59:50.385323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:01:40.690852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.395796+0000","last_change":"2023-10-07T07:59:49.373482+0000","last_active":"2023-10-07T07:59:50.395796+0000","last_peered":"2023-10-07T07:59:50.395796+0000","last_clean":"2023-10-07T07:59:50.395796+0000","last_became_active":"2023-10-07T07:59:49.373207+0000","last_became_peered":"2023-10-07T07:59:49.373207+0000","last_unstale":"2023-10-07T07:59:50.395796+0000","last_undegraded":"2023-10-07T07:59:50.395796+0000","last_fullsized":"2023-10-07T07:59:50.395796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:56:36.671145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388182+0000","last_change":"2023-10-07T07:59:50.379979+0000","last_active":"2023-10-07T07:59:50.388182+0000","last_peered":"2023-10-07T07:59:50.388182+0000","last_clean":"2023-10-07T07:59:50.388182+0000","last_became_active":"2023-10-07T07:59:50.379698+0000","last_became_peered":"2023-10-07T07:59:50.379698+0000","last_unstale":"2023-10-07T07:59:50.388182+0000","last_undegraded":"2023-10-07T07:59:50.388182+0000","last_fullsized":"2023-10-07T07:59:50.388182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:56:45.697716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.398429+0000","last_change":"2023-10-07T07:59:49.379035+0000","last_active":"2023-10-07T07:59:50.398429+0000","last_peered":"2023-10-07T07:59:50.398429+0000","last_clean":"2023-10-07T07:59:50.398429+0000","last_became_active":"2023-10-07T07:59:49.378771+0000","last_became_peered":"2023-10-07T07:59:49.378771+0000","last_unstale":"2023-10-07T07:59:50.398429+0000","last_undegraded":"2023-10-07T07:59:50.398429+0000","last_fullsized":"2023-10-07T07:59:50.398429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:45:08.759146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.387653+0000","last_change":"2023-10-07T07:59:50.377504+0000","last_active":"2023-10-07T07:59:50.387653+0000","last_peered":"2023-10-07T07:59:50.387653+0000","last_clean":"2023-10-07T07:59:50.387653+0000","last_became_active":"2023-10-07T07:59:50.377166+0000","last_became_peered":"2023-10-07T07:59:50.377166+0000","last_unstale":"2023-10-07T07:59:50.387653+0000","last_undegraded":"2023-10-07T07:59:50.387653+0000","last_fullsized":"2023-10-07T07:59:50.387653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:45:04.782948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.391361+0000","last_change":"2023-10-07T07:59:49.377233+0000","last_active":"2023-10-07T07:59:50.391361+0000","last_peered":"2023-10-07T07:59:50.391361+0000","last_clean":"2023-10-07T07:59:50.391361+0000","last_became_active":"2023-10-07T07:59:49.376890+0000","last_became_peered":"2023-10-07T07:59:49.376890+0000","last_unstale":"2023-10-07T07:59:50.391361+0000","last_undegraded":"2023-10-07T07:59:50.391361+0000","last_fullsized":"2023-10-07T07:59:50.391361+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:59:08.038668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382515+0000","last_change":"2023-10-07T07:59:50.376513+0000","last_active":"2023-10-07T07:59:50.382515+0000","last_peered":"2023-10-07T07:59:50.382515+0000","last_clean":"2023-10-07T07:59:50.382515+0000","last_became_active":"2023-10-07T07:59:50.376084+0000","last_became_peered":"2023-10-07T07:59:50.376084+0000","last_unstale":"2023-10-07T07:59:50.382515+0000","last_undegraded":"2023-10-07T07:59:50.382515+0000","last_fullsized":"2023-10-07T07:59:50.382515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:56:37.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.372388+0000","last_change":"2023-10-07T07:59:49.376228+0000","last_active":"2023-10-07T07:59:50.372388+0000","last_peered":"2023-10-07T07:59:50.372388+0000","last_clean":"2023-10-07T07:59:50.372388+0000","last_became_active":"2023-10-07T07:59:49.375942+0000","last_became_peered":"2023-10-07T07:59:49.375942+0000","last_unstale":"2023-10-07T07:59:50.372388+0000","last_undegraded":"2023-10-07T07:59:50.372388+0000","last_fullsized":"2023-10-07T07:59:50.372388+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:11:49.934509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382478+0000","last_change":"2023-10-07T07:59:50.376298+0000","last_active":"2023-10-07T07:59:50.382478+0000","last_peered":"2023-10-07T07:59:50.382478+0000","last_clean":"2023-10-07T07:59:50.382478+0000","last_became_active":"2023-10-07T07:59:50.376053+0000","last_became_peered":"2023-10-07T07:59:50.376053+0000","last_unstale":"2023-10-07T07:59:50.382478+0000","last_undegraded":"2023-10-07T07:59:50.382478+0000","last_fullsized":"2023-10-07T07:59:50.382478+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:56:18.818754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.362479+0000","last_change":"2023-10-07T07:59:49.372597+0000","last_active":"2023-10-07T07:59:50.362479+0000","last_peered":"2023-10-07T07:59:50.362479+0000","last_clean":"2023-10-07T07:59:50.362479+0000","last_became_active":"2023-10-07T07:59:49.372348+0000","last_became_peered":"2023-10-07T07:59:49.372348+0000","last_unstale":"2023-10-07T07:59:50.362479+0000","last_undegraded":"2023-10-07T07:59:50.362479+0000","last_fullsized":"2023-10-07T07:59:50.362479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:56:36.671145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393302+0000","last_change":"2023-10-07T07:59:50.387467+0000","last_active":"2023-10-07T07:59:50.393302+0000","last_peered":"2023-10-07T07:59:50.393302+0000","last_clean":"2023-10-07T07:59:50.393302+0000","last_became_active":"2023-10-07T07:59:50.378787+0000","last_became_peered":"2023-10-07T07:59:50.378787+0000","last_unstale":"2023-10-07T07:59:50.393302+0000","last_undegraded":"2023-10-07T07:59:50.393302+0000","last_fullsized":"2023-10-07T07:59:50.393302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:44:24.419529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394876+0000","last_change":"2023-10-07T07:59:49.380457+0000","last_active":"2023-10-07T07:59:50.394876+0000","last_peered":"2023-10-07T07:59:50.394876+0000","last_clean":"2023-10-07T07:59:50.394876+0000","last_became_active":"2023-10-07T07:59:49.380234+0000","last_became_peered":"2023-10-07T07:59:49.380234+0000","last_unstale":"2023-10-07T07:59:50.394876+0000","last_undegraded":"2023-10-07T07:59:50.394876+0000","last_fullsized":"2023-10-07T07:59:50.394876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:37:27.883546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.383426+0000","last_change":"2023-10-07T07:59:50.378317+0000","last_active":"2023-10-07T07:59:50.383426+0000","last_peered":"2023-10-07T07:59:50.383426+0000","last_clean":"2023-10-07T07:59:50.383426+0000","last_became_active":"2023-10-07T07:59:50.378065+0000","last_became_peered":"2023-10-07T07:59:50.378065+0000","last_unstale":"2023-10-07T07:59:50.383426+0000","last_undegraded":"2023-10-07T07:59:50.383426+0000","last_fullsized":"2023-10-07T07:59:50.383426+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:30.505062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.391809+0000","last_change":"2023-10-07T07:59:49.371088+0000","last_active":"2023-10-07T07:59:50.391809+0000","last_peered":"2023-10-07T07:59:50.391809+0000","last_clean":"2023-10-07T07:59:50.391809+0000","last_became_active":"2023-10-07T07:59:49.370795+0000","last_became_peered":"2023-10-07T07:59:49.370795+0000","last_unstale":"2023-10-07T07:59:50.391809+0000","last_undegraded":"2023-10-07T07:59:50.391809+0000","last_fullsized":"2023-10-07T07:59:50.391809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:12.440274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382598+0000","last_change":"2023-10-07T07:59:50.377264+0000","last_active":"2023-10-07T07:59:50.382598+0000","last_peered":"2023-10-07T07:59:50.382598+0000","last_clean":"2023-10-07T07:59:50.382598+0000","last_became_active":"2023-10-07T07:59:50.376948+0000","last_became_peered":"2023-10-07T07:59:50.376948+0000","last_unstale":"2023-10-07T07:59:50.382598+0000","last_undegraded":"2023-10-07T07:59:50.382598+0000","last_fullsized":"2023-10-07T07:59:50.382598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:29:31.121399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390675+0000","last_change":"2023-10-07T07:59:49.378173+0000","last_active":"2023-10-07T07:59:50.390675+0000","last_peered":"2023-10-07T07:59:50.390675+0000","last_clean":"2023-10-07T07:59:50.390675+0000","last_became_active":"2023-10-07T07:59:49.377874+0000","last_became_peered":"2023-10-07T07:59:49.377874+0000","last_unstale":"2023-10-07T07:59:50.390675+0000","last_undegraded":"2023-10-07T07:59:50.390675+0000","last_fullsized":"2023-10-07T07:59:50.390675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:39:26.601574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.391755+0000","last_change":"2023-10-07T07:59:50.378420+0000","last_active":"2023-10-07T07:59:50.391755+0000","last_peered":"2023-10-07T07:59:50.391755+0000","last_clean":"2023-10-07T07:59:50.391755+0000","last_became_active":"2023-10-07T07:59:50.376076+0000","last_became_peered":"2023-10-07T07:59:50.376076+0000","last_unstale":"2023-10-07T07:59:50.391755+0000","last_undegraded":"2023-10-07T07:59:50.391755+0000","last_fullsized":"2023-10-07T07:59:50.391755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:43:47.947126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.395533+0000","last_change":"2023-10-07T07:59:49.381767+0000","last_active":"2023-10-07T07:59:50.395533+0000","last_peered":"2023-10-07T07:59:50.395533+0000","last_clean":"2023-10-07T07:59:50.395533+0000","last_became_active":"2023-10-07T07:59:49.381461+0000","last_became_peered":"2023-10-07T07:59:49.381461+0000","last_unstale":"2023-10-07T07:59:50.395533+0000","last_undegraded":"2023-10-07T07:59:50.395533+0000","last_fullsized":"2023-10-07T07:59:50.395533+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:36:38.169887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.389026+0000","last_change":"2023-10-07T07:59:50.383903+0000","last_active":"2023-10-07T07:59:50.389026+0000","last_peered":"2023-10-07T07:59:50.389026+0000","last_clean":"2023-10-07T07:59:50.389026+0000","last_became_active":"2023-10-07T07:59:50.380146+0000","last_became_peered":"2023-10-07T07:59:50.380146+0000","last_unstale":"2023-10-07T07:59:50.389026+0000","last_undegraded":"2023-10-07T07:59:50.389026+0000","last_fullsized":"2023-10-07T07:59:50.389026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:12:17.784611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.362280+0000","last_change":"2023-10-07T07:59:49.373546+0000","last_active":"2023-10-07T07:59:50.362280+0000","last_peered":"2023-10-07T07:59:50.362280+0000","last_clean":"2023-10-07T07:59:50.362280+0000","last_became_active":"2023-10-07T07:59:49.373278+0000","last_became_peered":"2023-10-07T07:59:49.373278+0000","last_unstale":"2023-10-07T07:59:50.362280+0000","last_undegraded":"2023-10-07T07:59:50.362280+0000","last_fullsized":"2023-10-07T07:59:50.362280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:33:34.415390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390444+0000","last_change":"2023-10-07T07:59:50.381006+0000","last_active":"2023-10-07T07:59:50.390444+0000","last_peered":"2023-10-07T07:59:50.390444+0000","last_clean":"2023-10-07T07:59:50.390444+0000","last_became_active":"2023-10-07T07:59:50.380510+0000","last_became_peered":"2023-10-07T07:59:50.380510+0000","last_unstale":"2023-10-07T07:59:50.390444+0000","last_undegraded":"2023-10-07T07:59:50.390444+0000","last_fullsized":"2023-10-07T07:59:50.390444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:27:54.622462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.391656+0000","last_change":"2023-10-07T07:59:49.374719+0000","last_active":"2023-10-07T07:59:50.391656+0000","last_peered":"2023-10-07T07:59:50.391656+0000","last_clean":"2023-10-07T07:59:50.391656+0000","last_became_active":"2023-10-07T07:59:49.374337+0000","last_became_peered":"2023-10-07T07:59:49.374337+0000","last_unstale":"2023-10-07T07:59:50.391656+0000","last_undegraded":"2023-10-07T07:59:50.391656+0000","last_fullsized":"2023-10-07T07:59:50.391656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:20:59.576519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.383261+0000","last_change":"2023-10-07T07:59:50.376592+0000","last_active":"2023-10-07T07:59:50.383261+0000","last_peered":"2023-10-07T07:59:50.383261+0000","last_clean":"2023-10-07T07:59:50.383261+0000","last_became_active":"2023-10-07T07:59:50.376318+0000","last_became_peered":"2023-10-07T07:59:50.376318+0000","last_unstale":"2023-10-07T07:59:50.383261+0000","last_undegraded":"2023-10-07T07:59:50.383261+0000","last_fullsized":"2023-10-07T07:59:50.383261+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:40:45.957885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.372516+0000","last_change":"2023-10-07T07:59:49.372259+0000","last_active":"2023-10-07T07:59:50.372516+0000","last_peered":"2023-10-07T07:59:50.372516+0000","last_clean":"2023-10-07T07:59:50.372516+0000","last_became_active":"2023-10-07T07:59:49.371954+0000","last_became_peered":"2023-10-07T07:59:49.371954+0000","last_unstale":"2023-10-07T07:59:50.372516+0000","last_undegraded":"2023-10-07T07:59:50.372516+0000","last_fullsized":"2023-10-07T07:59:50.372516+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:22:25.688208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394167+0000","last_change":"2023-10-07T07:59:50.388442+0000","last_active":"2023-10-07T07:59:50.394167+0000","last_peered":"2023-10-07T07:59:50.394167+0000","last_clean":"2023-10-07T07:59:50.394167+0000","last_became_active":"2023-10-07T07:59:50.388068+0000","last_became_peered":"2023-10-07T07:59:50.388068+0000","last_unstale":"2023-10-07T07:59:50.394167+0000","last_undegraded":"2023-10-07T07:59:50.394167+0000","last_fullsized":"2023-10-07T07:59:50.394167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:43.243398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.373519+0000","last_change":"2023-10-07T07:59:49.374564+0000","last_active":"2023-10-07T07:59:50.373519+0000","last_peered":"2023-10-07T07:59:50.373519+0000","last_clean":"2023-10-07T07:59:50.373519+0000","last_became_active":"2023-10-07T07:59:49.374269+0000","last_became_peered":"2023-10-07T07:59:49.374269+0000","last_unstale":"2023-10-07T07:59:50.373519+0000","last_undegraded":"2023-10-07T07:59:50.373519+0000","last_fullsized":"2023-10-07T07:59:50.373519+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:12.440274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384501+0000","last_change":"2023-10-07T07:59:50.377812+0000","last_active":"2023-10-07T07:59:50.384501+0000","last_peered":"2023-10-07T07:59:50.384501+0000","last_clean":"2023-10-07T07:59:50.384501+0000","last_became_active":"2023-10-07T07:59:50.377017+0000","last_became_peered":"2023-10-07T07:59:50.377017+0000","last_unstale":"2023-10-07T07:59:50.384501+0000","last_undegraded":"2023-10-07T07:59:50.384501+0000","last_fullsized":"2023-10-07T07:59:50.384501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:27:54.622462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.369016+0000","last_change":"2023-10-07T07:59:49.376243+0000","last_active":"2023-10-07T07:59:50.369016+0000","last_peered":"2023-10-07T07:59:50.369016+0000","last_clean":"2023-10-07T07:59:50.369016+0000","last_became_active":"2023-10-07T07:59:49.375968+0000","last_became_peered":"2023-10-07T07:59:49.375968+0000","last_unstale":"2023-10-07T07:59:50.369016+0000","last_undegraded":"2023-10-07T07:59:50.369016+0000","last_fullsized":"2023-10-07T07:59:50.369016+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:19:57.632171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388223+0000","last_change":"2023-10-07T07:59:50.380281+0000","last_active":"2023-10-07T07:59:50.388223+0000","last_peered":"2023-10-07T07:59:50.388223+0000","last_clean":"2023-10-07T07:59:50.388223+0000","last_became_active":"2023-10-07T07:59:50.378933+0000","last_became_peered":"2023-10-07T07:59:50.378933+0000","last_unstale":"2023-10-07T07:59:50.388223+0000","last_undegraded":"2023-10-07T07:59:50.388223+0000","last_fullsized":"2023-10-07T07:59:50.388223+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:47:13.496115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.367713+0000","last_change":"2023-10-07T07:59:49.371241+0000","last_active":"2023-10-07T07:59:50.367713+0000","last_peered":"2023-10-07T07:59:50.367713+0000","last_clean":"2023-10-07T07:59:50.367713+0000","last_became_active":"2023-10-07T07:59:49.370957+0000","last_became_peered":"2023-10-07T07:59:49.370957+0000","last_unstale":"2023-10-07T07:59:50.367713+0000","last_undegraded":"2023-10-07T07:59:50.367713+0000","last_fullsized":"2023-10-07T07:59:50.367713+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:40:44.953040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390557+0000","last_change":"2023-10-07T07:59:50.383939+0000","last_active":"2023-10-07T07:59:50.390557+0000","last_peered":"2023-10-07T07:59:50.390557+0000","last_clean":"2023-10-07T07:59:50.390557+0000","last_became_active":"2023-10-07T07:59:50.383631+0000","last_became_peered":"2023-10-07T07:59:50.383631+0000","last_unstale":"2023-10-07T07:59:50.390557+0000","last_undegraded":"2023-10-07T07:59:50.390557+0000","last_fullsized":"2023-10-07T07:59:50.390557+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:22:45.457050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.372632+0000","last_change":"2023-10-07T07:59:49.373555+0000","last_active":"2023-10-07T07:59:50.372632+0000","last_peered":"2023-10-07T07:59:50.372632+0000","last_clean":"2023-10-07T07:59:50.372632+0000","last_became_active":"2023-10-07T07:59:49.373228+0000","last_became_peered":"2023-10-07T07:59:49.373228+0000","last_unstale":"2023-10-07T07:59:50.372632+0000","last_undegraded":"2023-10-07T07:59:50.372632+0000","last_fullsized":"2023-10-07T07:59:50.372632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:59:54.154634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388157+0000","last_change":"2023-10-07T07:59:50.379372+0000","last_active":"2023-10-07T07:59:50.388157+0000","last_peered":"2023-10-07T07:59:50.388157+0000","last_clean":"2023-10-07T07:59:50.388157+0000","last_became_active":"2023-10-07T07:59:50.378835+0000","last_became_peered":"2023-10-07T07:59:50.378835+0000","last_unstale":"2023-10-07T07:59:50.388157+0000","last_undegraded":"2023-10-07T07:59:50.388157+0000","last_fullsized":"2023-10-07T07:59:50.388157+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:57:00.460393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.362102+0000","last_change":"2023-10-07T07:59:49.374898+0000","last_active":"2023-10-07T07:59:50.362102+0000","last_peered":"2023-10-07T07:59:50.362102+0000","last_clean":"2023-10-07T07:59:50.362102+0000","last_became_active":"2023-10-07T07:59:49.374639+0000","last_became_peered":"2023-10-07T07:59:49.374639+0000","last_unstale":"2023-10-07T07:59:50.362102+0000","last_undegraded":"2023-10-07T07:59:50.362102+0000","last_fullsized":"2023-10-07T07:59:50.362102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:16:16.654046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390145+0000","last_change":"2023-10-07T07:59:50.380829+0000","last_active":"2023-10-07T07:59:50.390145+0000","last_peered":"2023-10-07T07:59:50.390145+0000","last_clean":"2023-10-07T07:59:50.390145+0000","last_became_active":"2023-10-07T07:59:50.380110+0000","last_became_peered":"2023-10-07T07:59:50.380110+0000","last_unstale":"2023-10-07T07:59:50.390145+0000","last_undegraded":"2023-10-07T07:59:50.390145+0000","last_fullsized":"2023-10-07T07:59:50.390145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:40:41.701801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.373150+0000","last_change":"2023-10-07T07:59:49.374567+0000","last_active":"2023-10-07T07:59:50.373150+0000","last_peered":"2023-10-07T07:59:50.373150+0000","last_clean":"2023-10-07T07:59:50.373150+0000","last_became_active":"2023-10-07T07:59:49.374099+0000","last_became_peered":"2023-10-07T07:59:49.374099+0000","last_unstale":"2023-10-07T07:59:50.373150+0000","last_undegraded":"2023-10-07T07:59:50.373150+0000","last_fullsized":"2023-10-07T07:59:50.373150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:17:51.045372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384479+0000","last_change":"2023-10-07T07:59:50.376601+0000","last_active":"2023-10-07T07:59:50.384479+0000","last_peered":"2023-10-07T07:59:50.384479+0000","last_clean":"2023-10-07T07:59:50.384479+0000","last_became_active":"2023-10-07T07:59:50.376128+0000","last_became_peered":"2023-10-07T07:59:50.376128+0000","last_unstale":"2023-10-07T07:59:50.384479+0000","last_undegraded":"2023-10-07T07:59:50.384479+0000","last_fullsized":"2023-10-07T07:59:50.384479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:25:31.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.367364+0000","last_change":"2023-10-07T07:59:49.382089+0000","last_active":"2023-10-07T07:59:50.367364+0000","last_peered":"2023-10-07T07:59:50.367364+0000","last_clean":"2023-10-07T07:59:50.367364+0000","last_became_active":"2023-10-07T07:59:49.381776+0000","last_became_peered":"2023-10-07T07:59:49.381776+0000","last_unstale":"2023-10-07T07:59:50.367364+0000","last_undegraded":"2023-10-07T07:59:50.367364+0000","last_fullsized":"2023-10-07T07:59:50.367364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:40:47.150577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390325+0000","last_change":"2023-10-07T07:59:50.387518+0000","last_active":"2023-10-07T07:59:50.390325+0000","last_peered":"2023-10-07T07:59:50.390325+0000","last_clean":"2023-10-07T07:59:50.390325+0000","last_became_active":"2023-10-07T07:59:50.387180+0000","last_became_peered":"2023-10-07T07:59:50.387180+0000","last_unstale":"2023-10-07T07:59:50.390325+0000","last_undegraded":"2023-10-07T07:59:50.390325+0000","last_fullsized":"2023-10-07T07:59:50.390325+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:30.505062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.362584+0000","last_change":"2023-10-07T07:59:49.379340+0000","last_active":"2023-10-07T07:59:50.362584+0000","last_peered":"2023-10-07T07:59:50.362584+0000","last_clean":"2023-10-07T07:59:50.362584+0000","last_became_active":"2023-10-07T07:59:49.379020+0000","last_became_peered":"2023-10-07T07:59:49.379020+0000","last_unstale":"2023-10-07T07:59:50.362584+0000","last_undegraded":"2023-10-07T07:59:50.362584+0000","last_fullsized":"2023-10-07T07:59:50.362584+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:01:01.579396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393324+0000","last_change":"2023-10-07T07:59:50.389623+0000","last_active":"2023-10-07T07:59:50.393324+0000","last_peered":"2023-10-07T07:59:50.393324+0000","last_clean":"2023-10-07T07:59:50.393324+0000","last_became_active":"2023-10-07T07:59:50.389398+0000","last_became_peered":"2023-10-07T07:59:50.389398+0000","last_unstale":"2023-10-07T07:59:50.393324+0000","last_undegraded":"2023-10-07T07:59:50.393324+0000","last_fullsized":"2023-10-07T07:59:50.393324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:21:00.581364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.363112+0000","last_change":"2023-10-07T07:59:49.379441+0000","last_active":"2023-10-07T07:59:50.363112+0000","last_peered":"2023-10-07T07:59:50.363112+0000","last_clean":"2023-10-07T07:59:50.363112+0000","last_became_active":"2023-10-07T07:59:49.379158+0000","last_became_peered":"2023-10-07T07:59:49.379158+0000","last_unstale":"2023-10-07T07:59:50.363112+0000","last_undegraded":"2023-10-07T07:59:50.363112+0000","last_fullsized":"2023-10-07T07:59:50.363112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:39:26.601574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384987+0000","last_change":"2023-10-07T07:59:50.379732+0000","last_active":"2023-10-07T07:59:50.384987+0000","last_peered":"2023-10-07T07:59:50.384987+0000","last_clean":"2023-10-07T07:59:50.384987+0000","last_became_active":"2023-10-07T07:59:50.379454+0000","last_became_peered":"2023-10-07T07:59:50.379454+0000","last_unstale":"2023-10-07T07:59:50.384987+0000","last_undegraded":"2023-10-07T07:59:50.384987+0000","last_fullsized":"2023-10-07T07:59:50.384987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:19:52.829382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393017+0000","last_change":"2023-10-07T07:59:49.375635+0000","last_active":"2023-10-07T07:59:50.393017+0000","last_peered":"2023-10-07T07:59:50.393017+0000","last_clean":"2023-10-07T07:59:50.393017+0000","last_became_active":"2023-10-07T07:59:49.375348+0000","last_became_peered":"2023-10-07T07:59:49.375348+0000","last_unstale":"2023-10-07T07:59:50.393017+0000","last_undegraded":"2023-10-07T07:59:50.393017+0000","last_fullsized":"2023-10-07T07:59:50.393017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:52:48.738067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.376223+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T07:59:50.376223+0000","last_peered":"2023-10-07T07:59:50.376223+0000","last_clean":"2023-10-07T07:59:50.376223+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T07:59:50.376223+0000","last_undegraded":"2023-10-07T07:59:50.376223+0000","last_fullsized":"2023-10-07T07:59:50.376223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297808,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074032,"statfs":{"total":96636764160,"available":96331808768,"internally_reserved":0,"allocated":532480,"data_stored":213317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.79800000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64300000000000002}]},{"osd":1,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":1.127,"15min":1.127},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.74299999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.50800000000000001}]},{"osd":2,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71099999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53500000000000003}]},{"osd":3,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.745,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67100000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.434},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67500000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55900000000000005}]},{"osd":5,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71199999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69699999999999995,"5min":1.048,"15min":1.048},"last":0.72299999999999998}]},{"osd":6,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.748,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72699999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.80300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297812,"kb_used_data":524,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074028,"statfs":{"total":96636764160,"available":96331804672,"internally_reserved":0,"allocated":536576,"data_stored":213871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70099999999999996}]},{"osd":1,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77700000000000002,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.68700000000000006,"15min":0.69799999999999995},"min":{"1min":0.39500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.76800000000000002,"15min":0.79500000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.85099999999999998,"5min":2.653,"15min":2.653},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88100000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.72499999999999998,"15min":0.747},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72099999999999997,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72399999999999998}]},{"osd":4,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.505,"15min":0.51000000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.627}]},{"osd":5,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.55500000000000005,"15min":0.56799999999999995},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73899999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":803721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.46700000000000003,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56599999999999995,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61099999999999999,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.32700000000000001}]},{"osd":2,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.754,"15min":0.80600000000000005},"min":{"1min":0.28799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.746,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.71599999999999997,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67000000000000004,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.63800000000000001,"15min":0.66200000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69599999999999995,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86499999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96899999999999997,"5min":1.452,"15min":1.452},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.82999999999999996,"15min":0.85299999999999998},"min":{"1min":0.51900000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90800000000000003,"5min":3.3929999999999998,"15min":3.3929999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0029999999999999,"5min":1.546,"15min":1.546},"last":0.74199999999999999}]},{"osd":6,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.58399999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0209999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.86499999999999999,"15min":0.88700000000000001},"min":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0569999999999999,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.874,"15min":0.89600000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.94299999999999995,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0369999999999999,"5min":1.585,"15min":1.585},"last":0.70899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297812,"kb_used_data":524,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074028,"statfs":{"total":96636764160,"available":96331804672,"internally_reserved":0,"allocated":536576,"data_stored":213927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.501,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56699999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.438,"15min":0.443},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58499999999999996,"5min":1.298,"15min":1.298},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":1.073,"15min":1.073},"last":0.628}]},{"osd":3,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.63100000000000001,"5min":1.032,"15min":1.032},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.35499999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59699999999999998,"5min":1.141,"15min":1.141},"last":0.64000000000000001}]},{"osd":4,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69599999999999995,"15min":0.70099999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.81299999999999994,"5min":1.218,"15min":1.218},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.52900000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86699999999999999,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.91300000000000003}]},{"osd":5,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.876,"5min":1.375,"15min":1.375},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82799999999999996,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88800000000000001,"5min":1.401,"15min":1.401},"last":0.872},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90400000000000003,"5min":1.462,"15min":1.462},"last":0.90200000000000002}]},{"osd":7,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74399999999999999,"15min":0.746},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.86199999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.80100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297808,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074032,"statfs":{"total":96636764160,"available":96331808768,"internally_reserved":0,"allocated":532480,"data_stored":213373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.504,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40100000000000002,"15min":0.39900000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.31900000000000001}]},{"osd":1,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.377},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.49299999999999999,"5min":0.747,"15min":0.747},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.38300000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34100000000000003}]},{"osd":4,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64300000000000002,"15min":0.63800000000000001},"min":{"1min":0.56899999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.80300000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.54800000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83499999999999996,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.755}]},{"osd":5,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.49299999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.81200000000000006,"5min":1.51,"15min":1.51},"last":0.80200000000000005}]},{"osd":6,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.51000000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.67800000000000005}]},{"osd":7,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.504,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84199999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85599999999999998,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.81799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":540672,"data_stored":214819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.26900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.433,"15min":0.437},"min":{"1min":0.317,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.5,"5min":1.016,"15min":1.016},"last":0.64100000000000001}]},{"osd":1,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.46000000000000002,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.55600000000000005,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.502},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52500000000000002,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.57699999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.443}]},{"osd":2,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.51300000000000001,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.60399999999999998,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.61899999999999999,"5min":1.323,"15min":1.323},"last":0.59699999999999998}]},{"osd":4,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.52600000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.79000000000000004,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83899999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.84099999999999997,"5min":1.327,"15min":1.327},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.82899999999999996,"15min":0.85199999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.80900000000000005,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.79700000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.86099999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.81399999999999995,"15min":0.83099999999999996},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.82999999999999996,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.018}]},{"osd":7,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.81599999999999995,"5min":1.528,"15min":1.528},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.82399999999999995,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0369999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":214911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39900000000000002}]},{"osd":2,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46999999999999997}]},{"osd":3,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.749,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55600000000000005}]},{"osd":5,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.318,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65000000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72399999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.70599999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.40699999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73399999999999999,"5min":0.878,"15min":0.878},"last":0.62}]},{"osd":7,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64000000000000001,"15min":0.64700000000000002},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":803673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72299999999999998,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.70799999999999996,"15min":0.72499999999999998},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.85199999999999998,"5min":3.375,"15min":3.375},"last":0.67900000000000005}]},{"osd":1,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.36499999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73399999999999999,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.67000000000000004,"15min":0.68500000000000005},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.748,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.748,"15min":0.77200000000000002},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80200000000000005,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.88},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.78900000000000003,"15min":0.82399999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80900000000000005,"5min":5.2699999999999996,"15min":5.2699999999999996},"last":0.79100000000000004}]},{"osd":3,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.74299999999999999,"15min":0.77100000000000002},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79900000000000004,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.0569999999999999,"15min":1.133},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73399999999999999,"5min":10.644,"15min":10.644},"last":0.84299999999999997}]},{"osd":4,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72799999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":1.242,"15min":1.242},"last":0.54300000000000004}]},{"osd":6,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78300000000000003,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79100000000000004,"5min":1.123,"15min":1.123},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T07:59:54.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T07:59:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.733+0000 7f5274e0e700 1 Processor -- start 2023-10-07T07:59:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.734+0000 7f5274e0e700 1 -- start start 2023-10-07T07:59:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f5274e0e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52701592c0 0x7f527006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f5274e0e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5270159690 0x7f527006a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f5274e0e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52700cab80 0x7f52700cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f5274e0e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527006bb70 con 0x7f5270159690 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f5274e0e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52700cda10 con 0x7f52701592c0 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f5274e0e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527006af60 con 0x7f52700cab80 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5270159690 0x7f527006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5270159690 0x7f527006a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38554/0 (socket says 172.21.15.7:38554) 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526dd9b700 1 -- 172.21.15.7:0/1926688918 learned_addr learned my addr 172.21.15.7:0/1926688918 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526e59c700 1 --2- 172.21.15.7:0/1926688918 >> v2:172.21.15.172:3300/0 conn(0x7f52701592c0 0x7f527006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526ed9d700 1 --2- 172.21.15.7:0/1926688918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52700cab80 0x7f52700cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526dd9b700 1 -- 172.21.15.7:0/1926688918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52700cab80 msgr2=0x7f52700cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526dd9b700 1 --2- 172.21.15.7:0/1926688918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52700cab80 0x7f52700cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526dd9b700 1 -- 172.21.15.7:0/1926688918 >> v2:172.21.15.172:3300/0 conn(0x7f52701592c0 msgr2=0x7f527006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526dd9b700 1 --2- 172.21.15.7:0/1926688918 >> v2:172.21.15.172:3300/0 conn(0x7f52701592c0 0x7f527006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526dd9b700 1 -- 172.21.15.7:0/1926688918 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527006b240 con 0x7f5270159690 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526e59c700 1 --2- 172.21.15.7:0/1926688918 >> v2:172.21.15.172:3300/0 conn(0x7f52701592c0 0x7f527006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.735+0000 7f526dd9b700 1 --2- 172.21.15.7:0/1926688918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5270159690 0x7f527006a930 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f527006daf0 tx=0x7f5264009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a94f800e9f996cd server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f526d59a700 1 -- 172.21.15.7:0/1926688918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400b860 con 0x7f5270159690 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f526d59a700 1 -- 172.21.15.7:0/1926688918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526400b9c0 con 0x7f5270159690 2023-10-07T07:59:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f526d59a700 1 -- 172.21.15.7:0/1926688918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400bd60 con 0x7f5270159690 2023-10-07T07:59:54.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f5274e0e700 1 -- 172.21.15.7:0/1926688918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5270159690 msgr2=0x7f527006a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f5274e0e700 1 --2- 172.21.15.7:0/1926688918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5270159690 0x7f527006a930 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f527006daf0 tx=0x7f5264009250 comp rx=0 tx=0).stop 2023-10-07T07:59:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f5274e0e700 1 -- 172.21.15.7:0/1926688918 shutdown_connections 2023-10-07T07:59:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f5274e0e700 1 --2- 172.21.15.7:0/1926688918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52700cab80 0x7f52700cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f5274e0e700 1 --2- 172.21.15.7:0/1926688918 >> v2:172.21.15.172:3300/0 conn(0x7f52701592c0 0x7f527006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f5274e0e700 1 --2- 172.21.15.7:0/1926688918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5270159690 0x7f527006a930 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f5274e0e700 1 -- 172.21.15.7:0/1926688918 >> 172.21.15.7:0/1926688918 conn(0x7f5270068bd0 msgr2=0x7f52700ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f5274e0e700 1 -- 172.21.15.7:0/1926688918 shutdown_connections 2023-10-07T07:59:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.736+0000 7f5274e0e700 1 -- 172.21.15.7:0/1926688918 wait complete. 2023-10-07T07:59:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.737+0000 7f5274e0e700 1 Processor -- start 2023-10-07T07:59:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f5274e0e700 1 -- start start 2023-10-07T07:59:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f5274e0e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52700cab80 0x7f5270152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f5274e0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52701592c0 0x7f527015da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f5274e0e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5270159690 0x7f527015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f5274e0e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527015ee80 con 0x7f52701592c0 2023-10-07T07:59:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f5274e0e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52700ad8b0 con 0x7f52700cab80 2023-10-07T07:59:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f5274e0e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527015e5e0 con 0x7f5270159690 2023-10-07T07:59:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f526dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52701592c0 0x7f527015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f526dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52701592c0 0x7f527015da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38568/0 (socket says 172.21.15.7:38568) 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.738+0000 7f526dd9b700 1 -- 172.21.15.7:0/2932931599 learned_addr learned my addr 172.21.15.7:0/2932931599 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f526dd9b700 1 -- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5270159690 msgr2=0x7f527015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f526dd9b700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5270159690 0x7f527015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f526dd9b700 1 -- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52700cab80 msgr2=0x7f5270152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f526dd9b700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52700cab80 0x7f5270152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f526dd9b700 1 -- 172.21.15.7:0/2932931599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52640094f0 con 0x7f52701592c0 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f526dd9b700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52701592c0 0x7f527015da40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f52640090b0 tx=0x7f5264017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f525e7fc700 1 -- 172.21.15.7:0/2932931599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264024440 con 0x7f52701592c0 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f525e7fc700 1 -- 172.21.15.7:0/2932931599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52640245e0 con 0x7f52701592c0 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527015e8c0 con 0x7f52701592c0 2023-10-07T07:59:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f525e7fc700 1 -- 172.21.15.7:0/2932931599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52640249a0 con 0x7f52701592c0 2023-10-07T07:59:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.739+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527015ec50 con 0x7f52701592c0 2023-10-07T07:59:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.740+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527015a160 con 0x7f52701592c0 2023-10-07T07:59:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.741+0000 7f525e7fc700 1 -- 172.21.15.7:0/2932931599 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f526401a420 con 0x7f52701592c0 2023-10-07T07:59:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.741+0000 7f525e7fc700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5250066f30 0x7f52500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.741+0000 7f525e7fc700 1 -- 172.21.15.7:0/2932931599 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5264086570 con 0x7f52701592c0 2023-10-07T07:59:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.741+0000 7f526e59c700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5250066f30 0x7f52500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.742+0000 7f526e59c700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5250066f30 0x7f52500693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5260006fd0 tx=0x7f5260008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.745+0000 7f525e7fc700 1 -- 172.21.15.7:0/2932931599 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f527015a160 con 0x7f52701592c0 2023-10-07T07:59:54.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.944+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f527015a160 con 0x7f5250066f30 2023-10-07T07:59:54.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.953+0000 7f525e7fc700 1 -- 172.21.15.7:0/2932931599 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289526 (secure 0 0 0) 0x7f527015a160 con 0x7f5250066f30 2023-10-07T07:59:54.957 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:54.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5250066f30 msgr2=0x7f52500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5250066f30 0x7f52500693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5260006fd0 tx=0x7f5260008040 comp rx=0 tx=0).stop 2023-10-07T07:59:54.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52701592c0 msgr2=0x7f527015da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:54.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52701592c0 0x7f527015da40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f52640090b0 tx=0x7f5264017000 comp rx=0 tx=0).stop 2023-10-07T07:59:54.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 shutdown_connections 2023-10-07T07:59:54.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5250066f30 0x7f52500693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5270159690 0x7f527015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52700cab80 0x7f5270152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 --2- 172.21.15.7:0/2932931599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52701592c0 0x7f527015da40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:54.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 >> 172.21.15.7:0/2932931599 conn(0x7f5270068bd0 msgr2=0x7f52700a9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:54.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 shutdown_connections 2023-10-07T07:59:54.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:54.956+0000 7f5274e0e700 1 -- 172.21.15.7:0/2932931599 wait complete. 2023-10-07T07:59:54.960 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T07:59:54.972 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-10-07T07:59:54.055376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383656,"kb_used_data":5352,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591064,"statfs":{"total":773094113280,"available":770653249536,"internally_reserved":0,"allocated":5480448,"data_stored":2891612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.702572"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.392695+0000","last_change":"2023-10-07T07:59:50.391481+0000","last_active":"2023-10-07T07:59:50.392695+0000","last_peered":"2023-10-07T07:59:50.392695+0000","last_clean":"2023-10-07T07:59:50.392695+0000","last_became_active":"2023-10-07T07:59:50.390948+0000","last_became_peered":"2023-10-07T07:59:50.390948+0000","last_unstale":"2023-10-07T07:59:50.392695+0000","last_undegraded":"2023-10-07T07:59:50.392695+0000","last_fullsized":"2023-10-07T07:59:50.392695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:24:48.018565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393362+0000","last_change":"2023-10-07T07:59:50.389001+0000","last_active":"2023-10-07T07:59:50.393362+0000","last_peered":"2023-10-07T07:59:50.393362+0000","last_clean":"2023-10-07T07:59:50.393362+0000","last_became_active":"2023-10-07T07:59:50.385542+0000","last_became_peered":"2023-10-07T07:59:50.385542+0000","last_unstale":"2023-10-07T07:59:50.393362+0000","last_undegraded":"2023-10-07T07:59:50.393362+0000","last_fullsized":"2023-10-07T07:59:50.393362+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:24.742162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.389006+0000","last_change":"2023-10-07T07:59:50.383670+0000","last_active":"2023-10-07T07:59:50.389006+0000","last_peered":"2023-10-07T07:59:50.389006+0000","last_clean":"2023-10-07T07:59:50.389006+0000","last_became_active":"2023-10-07T07:59:50.383343+0000","last_became_peered":"2023-10-07T07:59:50.383343+0000","last_unstale":"2023-10-07T07:59:50.389006+0000","last_undegraded":"2023-10-07T07:59:50.389006+0000","last_fullsized":"2023-10-07T07:59:50.389006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:17:52.050217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382762+0000","last_change":"2023-10-07T07:59:50.374608+0000","last_active":"2023-10-07T07:59:50.382762+0000","last_peered":"2023-10-07T07:59:50.382762+0000","last_clean":"2023-10-07T07:59:50.382762+0000","last_became_active":"2023-10-07T07:59:50.374347+0000","last_became_peered":"2023-10-07T07:59:50.374347+0000","last_unstale":"2023-10-07T07:59:50.382762+0000","last_undegraded":"2023-10-07T07:59:50.382762+0000","last_fullsized":"2023-10-07T07:59:50.382762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:19:08.600628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382530+0000","last_change":"2023-10-07T07:59:50.379141+0000","last_active":"2023-10-07T07:59:50.382530+0000","last_peered":"2023-10-07T07:59:50.382530+0000","last_clean":"2023-10-07T07:59:50.382530+0000","last_became_active":"2023-10-07T07:59:50.378816+0000","last_became_peered":"2023-10-07T07:59:50.378816+0000","last_unstale":"2023-10-07T07:59:50.382530+0000","last_undegraded":"2023-10-07T07:59:50.382530+0000","last_fullsized":"2023-10-07T07:59:50.382530+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:27:58.785440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.377924+0000","last_change":"2023-10-07T07:59:50.370620+0000","last_active":"2023-10-07T07:59:50.377924+0000","last_peered":"2023-10-07T07:59:50.377924+0000","last_clean":"2023-10-07T07:59:50.377924+0000","last_became_active":"2023-10-07T07:59:50.370261+0000","last_became_peered":"2023-10-07T07:59:50.370261+0000","last_unstale":"2023-10-07T07:59:50.377924+0000","last_undegraded":"2023-10-07T07:59:50.377924+0000","last_fullsized":"2023-10-07T07:59:50.377924+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:24.742162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388206+0000","last_change":"2023-10-07T07:59:50.379891+0000","last_active":"2023-10-07T07:59:50.388206+0000","last_peered":"2023-10-07T07:59:50.388206+0000","last_clean":"2023-10-07T07:59:50.388206+0000","last_became_active":"2023-10-07T07:59:50.378815+0000","last_became_peered":"2023-10-07T07:59:50.378815+0000","last_unstale":"2023-10-07T07:59:50.388206+0000","last_undegraded":"2023-10-07T07:59:50.388206+0000","last_fullsized":"2023-10-07T07:59:50.388206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:12:33.961667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.385352+0000","last_change":"2023-10-07T07:59:50.380131+0000","last_active":"2023-10-07T07:59:50.385352+0000","last_peered":"2023-10-07T07:59:50.385352+0000","last_clean":"2023-10-07T07:59:50.385352+0000","last_became_active":"2023-10-07T07:59:50.379887+0000","last_became_peered":"2023-10-07T07:59:50.379887+0000","last_unstale":"2023-10-07T07:59:50.385352+0000","last_undegraded":"2023-10-07T07:59:50.385352+0000","last_fullsized":"2023-10-07T07:59:50.385352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:40:48.155422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.385351+0000","last_change":"2023-10-07T07:59:50.380389+0000","last_active":"2023-10-07T07:59:50.385351+0000","last_peered":"2023-10-07T07:59:50.385351+0000","last_clean":"2023-10-07T07:59:50.385351+0000","last_became_active":"2023-10-07T07:59:50.380091+0000","last_became_peered":"2023-10-07T07:59:50.380091+0000","last_unstale":"2023-10-07T07:59:50.385351+0000","last_undegraded":"2023-10-07T07:59:50.385351+0000","last_fullsized":"2023-10-07T07:59:50.385351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:13.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394204+0000","last_change":"2023-10-07T07:59:50.390182+0000","last_active":"2023-10-07T07:59:50.394204+0000","last_peered":"2023-10-07T07:59:50.394204+0000","last_clean":"2023-10-07T07:59:50.394204+0000","last_became_active":"2023-10-07T07:59:50.389924+0000","last_became_peered":"2023-10-07T07:59:50.389924+0000","last_unstale":"2023-10-07T07:59:50.394204+0000","last_undegraded":"2023-10-07T07:59:50.394204+0000","last_fullsized":"2023-10-07T07:59:50.394204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:01:40.690852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.378170+0000","last_change":"2023-10-07T07:59:50.373976+0000","last_active":"2023-10-07T07:59:50.378170+0000","last_peered":"2023-10-07T07:59:50.378170+0000","last_clean":"2023-10-07T07:59:50.378170+0000","last_became_active":"2023-10-07T07:59:50.370357+0000","last_became_peered":"2023-10-07T07:59:50.370357+0000","last_unstale":"2023-10-07T07:59:50.378170+0000","last_undegraded":"2023-10-07T07:59:50.378170+0000","last_fullsized":"2023-10-07T07:59:50.378170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:02:53.923228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388901+0000","last_change":"2023-10-07T07:59:50.375292+0000","last_active":"2023-10-07T07:59:50.388901+0000","last_peered":"2023-10-07T07:59:50.388901+0000","last_clean":"2023-10-07T07:59:50.388901+0000","last_became_active":"2023-10-07T07:59:50.374983+0000","last_became_peered":"2023-10-07T07:59:50.374983+0000","last_unstale":"2023-10-07T07:59:50.388901+0000","last_undegraded":"2023-10-07T07:59:50.388901+0000","last_fullsized":"2023-10-07T07:59:50.388901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:40:45.957885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.383246+0000","last_change":"2023-10-07T07:59:50.375624+0000","last_active":"2023-10-07T07:59:50.383246+0000","last_peered":"2023-10-07T07:59:50.383246+0000","last_clean":"2023-10-07T07:59:50.383246+0000","last_became_active":"2023-10-07T07:59:50.375350+0000","last_became_peered":"2023-10-07T07:59:50.375350+0000","last_unstale":"2023-10-07T07:59:50.383246+0000","last_undegraded":"2023-10-07T07:59:50.383246+0000","last_fullsized":"2023-10-07T07:59:50.383246+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:21:00.581364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384299+0000","last_change":"2023-10-07T07:59:50.376296+0000","last_active":"2023-10-07T07:59:50.384299+0000","last_peered":"2023-10-07T07:59:50.384299+0000","last_clean":"2023-10-07T07:59:50.384299+0000","last_became_active":"2023-10-07T07:59:50.376014+0000","last_became_peered":"2023-10-07T07:59:50.376014+0000","last_unstale":"2023-10-07T07:59:50.384299+0000","last_undegraded":"2023-10-07T07:59:50.384299+0000","last_fullsized":"2023-10-07T07:59:50.384299+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:07:24.396636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384541+0000","last_change":"2023-10-07T07:59:50.378694+0000","last_active":"2023-10-07T07:59:50.384541+0000","last_peered":"2023-10-07T07:59:50.384541+0000","last_clean":"2023-10-07T07:59:50.384541+0000","last_became_active":"2023-10-07T07:59:50.378277+0000","last_became_peered":"2023-10-07T07:59:50.378277+0000","last_unstale":"2023-10-07T07:59:50.384541+0000","last_undegraded":"2023-10-07T07:59:50.384541+0000","last_fullsized":"2023-10-07T07:59:50.384541+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:37:30.025946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.383983+0000","last_change":"2023-10-07T07:59:50.380265+0000","last_active":"2023-10-07T07:59:50.383983+0000","last_peered":"2023-10-07T07:59:50.383983+0000","last_clean":"2023-10-07T07:59:50.383983+0000","last_became_active":"2023-10-07T07:59:50.379966+0000","last_became_peered":"2023-10-07T07:59:50.379966+0000","last_unstale":"2023-10-07T07:59:50.383983+0000","last_undegraded":"2023-10-07T07:59:50.383983+0000","last_fullsized":"2023-10-07T07:59:50.383983+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:04:07.523857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394017+0000","last_change":"2023-10-07T07:59:50.389873+0000","last_active":"2023-10-07T07:59:50.394017+0000","last_peered":"2023-10-07T07:59:50.394017+0000","last_clean":"2023-10-07T07:59:50.394017+0000","last_became_active":"2023-10-07T07:59:50.388544+0000","last_became_peered":"2023-10-07T07:59:50.388544+0000","last_unstale":"2023-10-07T07:59:50.394017+0000","last_undegraded":"2023-10-07T07:59:50.394017+0000","last_fullsized":"2023-10-07T07:59:50.394017+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:13.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.386882+0000","last_change":"2023-10-07T07:59:50.377558+0000","last_active":"2023-10-07T07:59:50.386882+0000","last_peered":"2023-10-07T07:59:50.386882+0000","last_clean":"2023-10-07T07:59:50.386882+0000","last_became_active":"2023-10-07T07:59:50.377246+0000","last_became_peered":"2023-10-07T07:59:50.377246+0000","last_unstale":"2023-10-07T07:59:50.386882+0000","last_undegraded":"2023-10-07T07:59:50.386882+0000","last_fullsized":"2023-10-07T07:59:50.386882+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:01:02.584241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384898+0000","last_change":"2023-10-07T07:59:50.381205+0000","last_active":"2023-10-07T07:59:50.384898+0000","last_peered":"2023-10-07T07:59:50.384898+0000","last_clean":"2023-10-07T07:59:50.384898+0000","last_became_active":"2023-10-07T07:59:50.380889+0000","last_became_peered":"2023-10-07T07:59:50.380889+0000","last_unstale":"2023-10-07T07:59:50.384898+0000","last_undegraded":"2023-10-07T07:59:50.384898+0000","last_fullsized":"2023-10-07T07:59:50.384898+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:14:18.711278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390538+0000","last_change":"2023-10-07T07:59:50.383548+0000","last_active":"2023-10-07T07:59:50.390538+0000","last_peered":"2023-10-07T07:59:50.390538+0000","last_clean":"2023-10-07T07:59:50.390538+0000","last_became_active":"2023-10-07T07:59:50.383165+0000","last_became_peered":"2023-10-07T07:59:50.383165+0000","last_unstale":"2023-10-07T07:59:50.390538+0000","last_undegraded":"2023-10-07T07:59:50.390538+0000","last_fullsized":"2023-10-07T07:59:50.390538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:37:30.025946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384502+0000","last_change":"2023-10-07T07:59:50.376768+0000","last_active":"2023-10-07T07:59:50.384502+0000","last_peered":"2023-10-07T07:59:50.384502+0000","last_clean":"2023-10-07T07:59:50.384502+0000","last_became_active":"2023-10-07T07:59:50.376432+0000","last_became_peered":"2023-10-07T07:59:50.376432+0000","last_unstale":"2023-10-07T07:59:50.384502+0000","last_undegraded":"2023-10-07T07:59:50.384502+0000","last_fullsized":"2023-10-07T07:59:50.384502+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:19:58.637016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.378282+0000","last_change":"2023-10-07T07:59:50.370978+0000","last_active":"2023-10-07T07:59:50.378282+0000","last_peered":"2023-10-07T07:59:50.378282+0000","last_clean":"2023-10-07T07:59:50.378282+0000","last_became_active":"2023-10-07T07:59:50.370712+0000","last_became_peered":"2023-10-07T07:59:50.370712+0000","last_unstale":"2023-10-07T07:59:50.378282+0000","last_undegraded":"2023-10-07T07:59:50.378282+0000","last_fullsized":"2023-10-07T07:59:50.378282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:33:35.420235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.389619+0000","last_change":"2023-10-07T07:59:50.383265+0000","last_active":"2023-10-07T07:59:50.389619+0000","last_peered":"2023-10-07T07:59:50.389619+0000","last_clean":"2023-10-07T07:59:50.389619+0000","last_became_active":"2023-10-07T07:59:50.381574+0000","last_became_peered":"2023-10-07T07:59:50.381574+0000","last_unstale":"2023-10-07T07:59:50.389619+0000","last_undegraded":"2023-10-07T07:59:50.389619+0000","last_fullsized":"2023-10-07T07:59:50.389619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:44:26.921725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393424+0000","last_change":"2023-10-07T07:59:50.387676+0000","last_active":"2023-10-07T07:59:50.393424+0000","last_peered":"2023-10-07T07:59:50.393424+0000","last_clean":"2023-10-07T07:59:50.393424+0000","last_became_active":"2023-10-07T07:59:50.378692+0000","last_became_peered":"2023-10-07T07:59:50.378692+0000","last_unstale":"2023-10-07T07:59:50.393424+0000","last_undegraded":"2023-10-07T07:59:50.393424+0000","last_fullsized":"2023-10-07T07:59:50.393424+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:37:28.888391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.392671+0000","last_change":"2023-10-07T07:59:50.391186+0000","last_active":"2023-10-07T07:59:50.392671+0000","last_peered":"2023-10-07T07:59:50.392671+0000","last_clean":"2023-10-07T07:59:50.392671+0000","last_became_active":"2023-10-07T07:59:50.390585+0000","last_became_peered":"2023-10-07T07:59:50.390585+0000","last_unstale":"2023-10-07T07:59:50.392671+0000","last_undegraded":"2023-10-07T07:59:50.392671+0000","last_fullsized":"2023-10-07T07:59:50.392671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:37:28.888391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.378902+0000","last_change":"2023-10-07T07:59:50.371153+0000","last_active":"2023-10-07T07:59:50.378902+0000","last_peered":"2023-10-07T07:59:50.378902+0000","last_clean":"2023-10-07T07:59:50.378902+0000","last_became_active":"2023-10-07T07:59:50.370893+0000","last_became_peered":"2023-10-07T07:59:50.370893+0000","last_unstale":"2023-10-07T07:59:50.378902+0000","last_undegraded":"2023-10-07T07:59:50.378902+0000","last_fullsized":"2023-10-07T07:59:50.378902+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:22:45.457050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.395633+0000","last_change":"2023-10-07T07:59:50.392978+0000","last_active":"2023-10-07T07:59:50.395633+0000","last_peered":"2023-10-07T07:59:50.395633+0000","last_clean":"2023-10-07T07:59:50.395633+0000","last_became_active":"2023-10-07T07:59:50.392660+0000","last_became_peered":"2023-10-07T07:59:50.392660+0000","last_unstale":"2023-10-07T07:59:50.395633+0000","last_undegraded":"2023-10-07T07:59:50.395633+0000","last_fullsized":"2023-10-07T07:59:50.395633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:25:19.271682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382961+0000","last_change":"2023-10-07T07:59:50.376540+0000","last_active":"2023-10-07T07:59:50.382961+0000","last_peered":"2023-10-07T07:59:50.382961+0000","last_clean":"2023-10-07T07:59:50.382961+0000","last_became_active":"2023-10-07T07:59:50.375928+0000","last_became_peered":"2023-10-07T07:59:50.375928+0000","last_unstale":"2023-10-07T07:59:50.382961+0000","last_undegraded":"2023-10-07T07:59:50.382961+0000","last_fullsized":"2023-10-07T07:59:50.382961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:16:52.133679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.386779+0000","last_change":"2023-10-07T07:59:50.385727+0000","last_active":"2023-10-07T07:59:50.386779+0000","last_peered":"2023-10-07T07:59:50.386779+0000","last_clean":"2023-10-07T07:59:50.386779+0000","last_became_active":"2023-10-07T07:59:50.385469+0000","last_became_peered":"2023-10-07T07:59:50.385469+0000","last_unstale":"2023-10-07T07:59:50.386779+0000","last_undegraded":"2023-10-07T07:59:50.386779+0000","last_fullsized":"2023-10-07T07:59:50.386779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:43:44.111178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393628+0000","last_change":"2023-10-07T07:59:50.389947+0000","last_active":"2023-10-07T07:59:50.393628+0000","last_peered":"2023-10-07T07:59:50.393628+0000","last_clean":"2023-10-07T07:59:50.393628+0000","last_became_active":"2023-10-07T07:59:50.388636+0000","last_became_peered":"2023-10-07T07:59:50.388636+0000","last_unstale":"2023-10-07T07:59:50.393628+0000","last_undegraded":"2023-10-07T07:59:50.393628+0000","last_fullsized":"2023-10-07T07:59:50.393628+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:24:48.018565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384353+0000","last_change":"2023-10-07T07:59:50.379291+0000","last_active":"2023-10-07T07:59:50.384353+0000","last_peered":"2023-10-07T07:59:50.384353+0000","last_clean":"2023-10-07T07:59:50.384353+0000","last_became_active":"2023-10-07T07:59:50.378875+0000","last_became_peered":"2023-10-07T07:59:50.378875+0000","last_unstale":"2023-10-07T07:59:50.384353+0000","last_undegraded":"2023-10-07T07:59:50.384353+0000","last_fullsized":"2023-10-07T07:59:50.384353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:43:47.947126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.376488+0000","last_change":"2023-10-07T07:59:50.369149+0000","last_active":"2023-10-07T07:59:50.376488+0000","last_peered":"2023-10-07T07:59:50.376488+0000","last_clean":"2023-10-07T07:59:50.376488+0000","last_became_active":"2023-10-07T07:59:50.368914+0000","last_became_peered":"2023-10-07T07:59:50.368914+0000","last_unstale":"2023-10-07T07:59:50.376488+0000","last_undegraded":"2023-10-07T07:59:50.376488+0000","last_fullsized":"2023-10-07T07:59:50.376488+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:40:41.701801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382454+0000","last_change":"2023-10-07T07:59:50.377568+0000","last_active":"2023-10-07T07:59:50.382454+0000","last_peered":"2023-10-07T07:59:50.382454+0000","last_clean":"2023-10-07T07:59:50.382454+0000","last_became_active":"2023-10-07T07:59:50.377139+0000","last_became_peered":"2023-10-07T07:59:50.377139+0000","last_unstale":"2023-10-07T07:59:50.382454+0000","last_undegraded":"2023-10-07T07:59:50.382454+0000","last_fullsized":"2023-10-07T07:59:50.382454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:31:21.880962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.386265+0000","last_change":"2023-10-07T07:59:49.371877+0000","last_active":"2023-10-07T07:59:50.386265+0000","last_peered":"2023-10-07T07:59:50.386265+0000","last_clean":"2023-10-07T07:59:50.386265+0000","last_became_active":"2023-10-07T07:59:49.371606+0000","last_became_peered":"2023-10-07T07:59:49.371606+0000","last_unstale":"2023-10-07T07:59:50.386265+0000","last_undegraded":"2023-10-07T07:59:50.386265+0000","last_fullsized":"2023-10-07T07:59:50.386265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:29.500217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.386217+0000","last_change":"2023-10-07T07:59:50.384602+0000","last_active":"2023-10-07T07:59:50.386217+0000","last_peered":"2023-10-07T07:59:50.386217+0000","last_clean":"2023-10-07T07:59:50.386217+0000","last_became_active":"2023-10-07T07:59:50.384255+0000","last_became_peered":"2023-10-07T07:59:50.384255+0000","last_unstale":"2023-10-07T07:59:50.386217+0000","last_undegraded":"2023-10-07T07:59:50.386217+0000","last_fullsized":"2023-10-07T07:59:50.386217+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:19:39.418430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393745+0000","last_change":"2023-10-07T07:59:49.380830+0000","last_active":"2023-10-07T07:59:50.393745+0000","last_peered":"2023-10-07T07:59:50.393745+0000","last_clean":"2023-10-07T07:59:50.393745+0000","last_became_active":"2023-10-07T07:59:49.380454+0000","last_became_peered":"2023-10-07T07:59:49.380454+0000","last_unstale":"2023-10-07T07:59:50.393745+0000","last_undegraded":"2023-10-07T07:59:50.393745+0000","last_fullsized":"2023-10-07T07:59:50.393745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:22:44.452205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393666+0000","last_change":"2023-10-07T07:59:50.389770+0000","last_active":"2023-10-07T07:59:50.393666+0000","last_peered":"2023-10-07T07:59:50.393666+0000","last_clean":"2023-10-07T07:59:50.393666+0000","last_became_active":"2023-10-07T07:59:50.385593+0000","last_became_peered":"2023-10-07T07:59:50.385593+0000","last_unstale":"2023-10-07T07:59:50.393666+0000","last_undegraded":"2023-10-07T07:59:50.393666+0000","last_fullsized":"2023-10-07T07:59:50.393666+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:45:09.763991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.385656+0000","last_change":"2023-10-07T07:59:49.373004+0000","last_active":"2023-10-07T07:59:50.385656+0000","last_peered":"2023-10-07T07:59:50.385656+0000","last_clean":"2023-10-07T07:59:50.385656+0000","last_became_active":"2023-10-07T07:59:49.371664+0000","last_became_peered":"2023-10-07T07:59:49.371664+0000","last_unstale":"2023-10-07T07:59:50.385656+0000","last_undegraded":"2023-10-07T07:59:50.385656+0000","last_fullsized":"2023-10-07T07:59:50.385656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:45:03.778103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384332+0000","last_change":"2023-10-07T07:59:50.375337+0000","last_active":"2023-10-07T07:59:50.384332+0000","last_peered":"2023-10-07T07:59:50.384332+0000","last_clean":"2023-10-07T07:59:50.384332+0000","last_became_active":"2023-10-07T07:59:50.375090+0000","last_became_peered":"2023-10-07T07:59:50.375090+0000","last_unstale":"2023-10-07T07:59:50.384332+0000","last_undegraded":"2023-10-07T07:59:50.384332+0000","last_fullsized":"2023-10-07T07:59:50.384332+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:52:49.742912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394274+0000","last_change":"2023-10-07T07:59:49.380751+0000","last_active":"2023-10-07T07:59:50.394274+0000","last_peered":"2023-10-07T07:59:50.394274+0000","last_clean":"2023-10-07T07:59:50.394274+0000","last_became_active":"2023-10-07T07:59:49.380422+0000","last_became_peered":"2023-10-07T07:59:49.380422+0000","last_unstale":"2023-10-07T07:59:50.394274+0000","last_undegraded":"2023-10-07T07:59:50.394274+0000","last_fullsized":"2023-10-07T07:59:50.394274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:02:52.918383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.392476+0000","last_change":"2023-10-07T07:59:50.383246+0000","last_active":"2023-10-07T07:59:50.392476+0000","last_peered":"2023-10-07T07:59:50.392476+0000","last_clean":"2023-10-07T07:59:50.392476+0000","last_became_active":"2023-10-07T07:59:50.382897+0000","last_became_peered":"2023-10-07T07:59:50.382897+0000","last_unstale":"2023-10-07T07:59:50.392476+0000","last_undegraded":"2023-10-07T07:59:50.392476+0000","last_fullsized":"2023-10-07T07:59:50.392476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:33:16.341323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.395405+0000","last_change":"2023-10-07T07:59:49.373658+0000","last_active":"2023-10-07T07:59:50.395405+0000","last_peered":"2023-10-07T07:59:50.395405+0000","last_clean":"2023-10-07T07:59:50.395405+0000","last_became_active":"2023-10-07T07:59:49.373384+0000","last_became_peered":"2023-10-07T07:59:49.373384+0000","last_unstale":"2023-10-07T07:59:50.395405+0000","last_undegraded":"2023-10-07T07:59:50.395405+0000","last_fullsized":"2023-10-07T07:59:50.395405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:31:20.876117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.389932+0000","last_change":"2023-10-07T07:59:50.379482+0000","last_active":"2023-10-07T07:59:50.389932+0000","last_peered":"2023-10-07T07:59:50.389932+0000","last_clean":"2023-10-07T07:59:50.389932+0000","last_became_active":"2023-10-07T07:59:50.379174+0000","last_became_peered":"2023-10-07T07:59:50.379174+0000","last_unstale":"2023-10-07T07:59:50.389932+0000","last_undegraded":"2023-10-07T07:59:50.389932+0000","last_fullsized":"2023-10-07T07:59:50.389932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:43.243398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.392949+0000","last_change":"2023-10-07T07:59:49.378611+0000","last_active":"2023-10-07T07:59:50.392949+0000","last_peered":"2023-10-07T07:59:50.392949+0000","last_clean":"2023-10-07T07:59:50.392949+0000","last_became_active":"2023-10-07T07:59:49.377794+0000","last_became_peered":"2023-10-07T07:59:49.377794+0000","last_unstale":"2023-10-07T07:59:50.392949+0000","last_undegraded":"2023-10-07T07:59:50.392949+0000","last_fullsized":"2023-10-07T07:59:50.392949+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:25:30.259592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390450+0000","last_change":"2023-10-07T07:59:50.381157+0000","last_active":"2023-10-07T07:59:50.390450+0000","last_peered":"2023-10-07T07:59:50.390450+0000","last_clean":"2023-10-07T07:59:50.390450+0000","last_became_active":"2023-10-07T07:59:50.380898+0000","last_became_peered":"2023-10-07T07:59:50.380898+0000","last_unstale":"2023-10-07T07:59:50.390450+0000","last_undegraded":"2023-10-07T07:59:50.390450+0000","last_fullsized":"2023-10-07T07:59:50.390450+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:07:24.396636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.376037+0000","last_change":"2023-10-07T07:59:49.376511+0000","last_active":"2023-10-07T07:59:50.376037+0000","last_peered":"2023-10-07T07:59:50.376037+0000","last_clean":"2023-10-07T07:59:50.376037+0000","last_became_active":"2023-10-07T07:59:49.376207+0000","last_became_peered":"2023-10-07T07:59:49.376207+0000","last_unstale":"2023-10-07T07:59:50.376037+0000","last_undegraded":"2023-10-07T07:59:50.376037+0000","last_fullsized":"2023-10-07T07:59:50.376037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:12:16.779766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388234+0000","last_change":"2023-10-07T07:59:50.379115+0000","last_active":"2023-10-07T07:59:50.388234+0000","last_peered":"2023-10-07T07:59:50.388234+0000","last_clean":"2023-10-07T07:59:50.388234+0000","last_became_active":"2023-10-07T07:59:50.378371+0000","last_became_peered":"2023-10-07T07:59:50.378371+0000","last_unstale":"2023-10-07T07:59:50.388234+0000","last_undegraded":"2023-10-07T07:59:50.388234+0000","last_fullsized":"2023-10-07T07:59:50.388234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:25:09.893988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.373624+0000","last_change":"2023-10-07T07:59:49.376140+0000","last_active":"2023-10-07T07:59:50.373624+0000","last_peered":"2023-10-07T07:59:50.373624+0000","last_clean":"2023-10-07T07:59:50.373624+0000","last_became_active":"2023-10-07T07:59:49.375871+0000","last_became_peered":"2023-10-07T07:59:49.375871+0000","last_unstale":"2023-10-07T07:59:50.373624+0000","last_undegraded":"2023-10-07T07:59:50.373624+0000","last_fullsized":"2023-10-07T07:59:50.373624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:24:47.013720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388768+0000","last_change":"2023-10-07T07:59:50.384640+0000","last_active":"2023-10-07T07:59:50.388768+0000","last_peered":"2023-10-07T07:59:50.388768+0000","last_clean":"2023-10-07T07:59:50.388768+0000","last_became_active":"2023-10-07T07:59:50.384336+0000","last_became_peered":"2023-10-07T07:59:50.384336+0000","last_unstale":"2023-10-07T07:59:50.388768+0000","last_undegraded":"2023-10-07T07:59:50.388768+0000","last_fullsized":"2023-10-07T07:59:50.388768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:33:35.420235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.371756+0000","last_change":"2023-10-07T07:59:49.375115+0000","last_active":"2023-10-07T07:59:50.371756+0000","last_peered":"2023-10-07T07:59:50.371756+0000","last_clean":"2023-10-07T07:59:50.371756+0000","last_became_active":"2023-10-07T07:59:49.374846+0000","last_became_peered":"2023-10-07T07:59:49.374846+0000","last_unstale":"2023-10-07T07:59:50.371756+0000","last_undegraded":"2023-10-07T07:59:50.371756+0000","last_fullsized":"2023-10-07T07:59:50.371756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:59:08.038668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388928+0000","last_change":"2023-10-07T07:59:50.384852+0000","last_active":"2023-10-07T07:59:50.388928+0000","last_peered":"2023-10-07T07:59:50.388928+0000","last_clean":"2023-10-07T07:59:50.388928+0000","last_became_active":"2023-10-07T07:59:50.384559+0000","last_became_peered":"2023-10-07T07:59:50.384559+0000","last_unstale":"2023-10-07T07:59:50.388928+0000","last_undegraded":"2023-10-07T07:59:50.388928+0000","last_fullsized":"2023-10-07T07:59:50.388928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:56:18.818754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.374777+0000","last_change":"2023-10-07T07:59:49.375370+0000","last_active":"2023-10-07T07:59:50.374777+0000","last_peered":"2023-10-07T07:59:50.374777+0000","last_clean":"2023-10-07T07:59:50.374777+0000","last_became_active":"2023-10-07T07:59:49.374821+0000","last_became_peered":"2023-10-07T07:59:49.374821+0000","last_unstale":"2023-10-07T07:59:50.374777+0000","last_undegraded":"2023-10-07T07:59:50.374777+0000","last_fullsized":"2023-10-07T07:59:50.374777+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:44:23.414684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.385323+0000","last_change":"2023-10-07T07:59:50.379896+0000","last_active":"2023-10-07T07:59:50.385323+0000","last_peered":"2023-10-07T07:59:50.385323+0000","last_clean":"2023-10-07T07:59:50.385323+0000","last_became_active":"2023-10-07T07:59:50.379598+0000","last_became_peered":"2023-10-07T07:59:50.379598+0000","last_unstale":"2023-10-07T07:59:50.385323+0000","last_undegraded":"2023-10-07T07:59:50.385323+0000","last_fullsized":"2023-10-07T07:59:50.385323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:01:40.690852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.395796+0000","last_change":"2023-10-07T07:59:49.373482+0000","last_active":"2023-10-07T07:59:50.395796+0000","last_peered":"2023-10-07T07:59:50.395796+0000","last_clean":"2023-10-07T07:59:50.395796+0000","last_became_active":"2023-10-07T07:59:49.373207+0000","last_became_peered":"2023-10-07T07:59:49.373207+0000","last_unstale":"2023-10-07T07:59:50.395796+0000","last_undegraded":"2023-10-07T07:59:50.395796+0000","last_fullsized":"2023-10-07T07:59:50.395796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:56:36.671145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388182+0000","last_change":"2023-10-07T07:59:50.379979+0000","last_active":"2023-10-07T07:59:50.388182+0000","last_peered":"2023-10-07T07:59:50.388182+0000","last_clean":"2023-10-07T07:59:50.388182+0000","last_became_active":"2023-10-07T07:59:50.379698+0000","last_became_peered":"2023-10-07T07:59:50.379698+0000","last_unstale":"2023-10-07T07:59:50.388182+0000","last_undegraded":"2023-10-07T07:59:50.388182+0000","last_fullsized":"2023-10-07T07:59:50.388182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T16:56:45.697716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.398429+0000","last_change":"2023-10-07T07:59:49.379035+0000","last_active":"2023-10-07T07:59:50.398429+0000","last_peered":"2023-10-07T07:59:50.398429+0000","last_clean":"2023-10-07T07:59:50.398429+0000","last_became_active":"2023-10-07T07:59:49.378771+0000","last_became_peered":"2023-10-07T07:59:49.378771+0000","last_unstale":"2023-10-07T07:59:50.398429+0000","last_undegraded":"2023-10-07T07:59:50.398429+0000","last_fullsized":"2023-10-07T07:59:50.398429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:45:08.759146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.387653+0000","last_change":"2023-10-07T07:59:50.377504+0000","last_active":"2023-10-07T07:59:50.387653+0000","last_peered":"2023-10-07T07:59:50.387653+0000","last_clean":"2023-10-07T07:59:50.387653+0000","last_became_active":"2023-10-07T07:59:50.377166+0000","last_became_peered":"2023-10-07T07:59:50.377166+0000","last_unstale":"2023-10-07T07:59:50.387653+0000","last_undegraded":"2023-10-07T07:59:50.387653+0000","last_fullsized":"2023-10-07T07:59:50.387653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:45:04.782948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.391361+0000","last_change":"2023-10-07T07:59:49.377233+0000","last_active":"2023-10-07T07:59:50.391361+0000","last_peered":"2023-10-07T07:59:50.391361+0000","last_clean":"2023-10-07T07:59:50.391361+0000","last_became_active":"2023-10-07T07:59:49.376890+0000","last_became_peered":"2023-10-07T07:59:49.376890+0000","last_unstale":"2023-10-07T07:59:50.391361+0000","last_undegraded":"2023-10-07T07:59:50.391361+0000","last_fullsized":"2023-10-07T07:59:50.391361+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:59:08.038668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382515+0000","last_change":"2023-10-07T07:59:50.376513+0000","last_active":"2023-10-07T07:59:50.382515+0000","last_peered":"2023-10-07T07:59:50.382515+0000","last_clean":"2023-10-07T07:59:50.382515+0000","last_became_active":"2023-10-07T07:59:50.376084+0000","last_became_peered":"2023-10-07T07:59:50.376084+0000","last_unstale":"2023-10-07T07:59:50.382515+0000","last_undegraded":"2023-10-07T07:59:50.382515+0000","last_fullsized":"2023-10-07T07:59:50.382515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:56:37.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.372388+0000","last_change":"2023-10-07T07:59:49.376228+0000","last_active":"2023-10-07T07:59:50.372388+0000","last_peered":"2023-10-07T07:59:50.372388+0000","last_clean":"2023-10-07T07:59:50.372388+0000","last_became_active":"2023-10-07T07:59:49.375942+0000","last_became_peered":"2023-10-07T07:59:49.375942+0000","last_unstale":"2023-10-07T07:59:50.372388+0000","last_undegraded":"2023-10-07T07:59:50.372388+0000","last_fullsized":"2023-10-07T07:59:50.372388+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:11:49.934509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382478+0000","last_change":"2023-10-07T07:59:50.376298+0000","last_active":"2023-10-07T07:59:50.382478+0000","last_peered":"2023-10-07T07:59:50.382478+0000","last_clean":"2023-10-07T07:59:50.382478+0000","last_became_active":"2023-10-07T07:59:50.376053+0000","last_became_peered":"2023-10-07T07:59:50.376053+0000","last_unstale":"2023-10-07T07:59:50.382478+0000","last_undegraded":"2023-10-07T07:59:50.382478+0000","last_fullsized":"2023-10-07T07:59:50.382478+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:56:18.818754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.362479+0000","last_change":"2023-10-07T07:59:49.372597+0000","last_active":"2023-10-07T07:59:50.362479+0000","last_peered":"2023-10-07T07:59:50.362479+0000","last_clean":"2023-10-07T07:59:50.362479+0000","last_became_active":"2023-10-07T07:59:49.372348+0000","last_became_peered":"2023-10-07T07:59:49.372348+0000","last_unstale":"2023-10-07T07:59:50.362479+0000","last_undegraded":"2023-10-07T07:59:50.362479+0000","last_fullsized":"2023-10-07T07:59:50.362479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:56:36.671145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393302+0000","last_change":"2023-10-07T07:59:50.387467+0000","last_active":"2023-10-07T07:59:50.393302+0000","last_peered":"2023-10-07T07:59:50.393302+0000","last_clean":"2023-10-07T07:59:50.393302+0000","last_became_active":"2023-10-07T07:59:50.378787+0000","last_became_peered":"2023-10-07T07:59:50.378787+0000","last_unstale":"2023-10-07T07:59:50.393302+0000","last_undegraded":"2023-10-07T07:59:50.393302+0000","last_fullsized":"2023-10-07T07:59:50.393302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:44:24.419529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394876+0000","last_change":"2023-10-07T07:59:49.380457+0000","last_active":"2023-10-07T07:59:50.394876+0000","last_peered":"2023-10-07T07:59:50.394876+0000","last_clean":"2023-10-07T07:59:50.394876+0000","last_became_active":"2023-10-07T07:59:49.380234+0000","last_became_peered":"2023-10-07T07:59:49.380234+0000","last_unstale":"2023-10-07T07:59:50.394876+0000","last_undegraded":"2023-10-07T07:59:50.394876+0000","last_fullsized":"2023-10-07T07:59:50.394876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:37:27.883546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.383426+0000","last_change":"2023-10-07T07:59:50.378317+0000","last_active":"2023-10-07T07:59:50.383426+0000","last_peered":"2023-10-07T07:59:50.383426+0000","last_clean":"2023-10-07T07:59:50.383426+0000","last_became_active":"2023-10-07T07:59:50.378065+0000","last_became_peered":"2023-10-07T07:59:50.378065+0000","last_unstale":"2023-10-07T07:59:50.383426+0000","last_undegraded":"2023-10-07T07:59:50.383426+0000","last_fullsized":"2023-10-07T07:59:50.383426+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:30.505062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.391809+0000","last_change":"2023-10-07T07:59:49.371088+0000","last_active":"2023-10-07T07:59:50.391809+0000","last_peered":"2023-10-07T07:59:50.391809+0000","last_clean":"2023-10-07T07:59:50.391809+0000","last_became_active":"2023-10-07T07:59:49.370795+0000","last_became_peered":"2023-10-07T07:59:49.370795+0000","last_unstale":"2023-10-07T07:59:50.391809+0000","last_undegraded":"2023-10-07T07:59:50.391809+0000","last_fullsized":"2023-10-07T07:59:50.391809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:12.440274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.382598+0000","last_change":"2023-10-07T07:59:50.377264+0000","last_active":"2023-10-07T07:59:50.382598+0000","last_peered":"2023-10-07T07:59:50.382598+0000","last_clean":"2023-10-07T07:59:50.382598+0000","last_became_active":"2023-10-07T07:59:50.376948+0000","last_became_peered":"2023-10-07T07:59:50.376948+0000","last_unstale":"2023-10-07T07:59:50.382598+0000","last_undegraded":"2023-10-07T07:59:50.382598+0000","last_fullsized":"2023-10-07T07:59:50.382598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:29:31.121399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390675+0000","last_change":"2023-10-07T07:59:49.378173+0000","last_active":"2023-10-07T07:59:50.390675+0000","last_peered":"2023-10-07T07:59:50.390675+0000","last_clean":"2023-10-07T07:59:50.390675+0000","last_became_active":"2023-10-07T07:59:49.377874+0000","last_became_peered":"2023-10-07T07:59:49.377874+0000","last_unstale":"2023-10-07T07:59:50.390675+0000","last_undegraded":"2023-10-07T07:59:50.390675+0000","last_fullsized":"2023-10-07T07:59:50.390675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:39:26.601574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.391755+0000","last_change":"2023-10-07T07:59:50.378420+0000","last_active":"2023-10-07T07:59:50.391755+0000","last_peered":"2023-10-07T07:59:50.391755+0000","last_clean":"2023-10-07T07:59:50.391755+0000","last_became_active":"2023-10-07T07:59:50.376076+0000","last_became_peered":"2023-10-07T07:59:50.376076+0000","last_unstale":"2023-10-07T07:59:50.391755+0000","last_undegraded":"2023-10-07T07:59:50.391755+0000","last_fullsized":"2023-10-07T07:59:50.391755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:43:47.947126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.395533+0000","last_change":"2023-10-07T07:59:49.381767+0000","last_active":"2023-10-07T07:59:50.395533+0000","last_peered":"2023-10-07T07:59:50.395533+0000","last_clean":"2023-10-07T07:59:50.395533+0000","last_became_active":"2023-10-07T07:59:49.381461+0000","last_became_peered":"2023-10-07T07:59:49.381461+0000","last_unstale":"2023-10-07T07:59:50.395533+0000","last_undegraded":"2023-10-07T07:59:50.395533+0000","last_fullsized":"2023-10-07T07:59:50.395533+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:36:38.169887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.389026+0000","last_change":"2023-10-07T07:59:50.383903+0000","last_active":"2023-10-07T07:59:50.389026+0000","last_peered":"2023-10-07T07:59:50.389026+0000","last_clean":"2023-10-07T07:59:50.389026+0000","last_became_active":"2023-10-07T07:59:50.380146+0000","last_became_peered":"2023-10-07T07:59:50.380146+0000","last_unstale":"2023-10-07T07:59:50.389026+0000","last_undegraded":"2023-10-07T07:59:50.389026+0000","last_fullsized":"2023-10-07T07:59:50.389026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:12:17.784611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.362280+0000","last_change":"2023-10-07T07:59:49.373546+0000","last_active":"2023-10-07T07:59:50.362280+0000","last_peered":"2023-10-07T07:59:50.362280+0000","last_clean":"2023-10-07T07:59:50.362280+0000","last_became_active":"2023-10-07T07:59:49.373278+0000","last_became_peered":"2023-10-07T07:59:49.373278+0000","last_unstale":"2023-10-07T07:59:50.362280+0000","last_undegraded":"2023-10-07T07:59:50.362280+0000","last_fullsized":"2023-10-07T07:59:50.362280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:33:34.415390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390444+0000","last_change":"2023-10-07T07:59:50.381006+0000","last_active":"2023-10-07T07:59:50.390444+0000","last_peered":"2023-10-07T07:59:50.390444+0000","last_clean":"2023-10-07T07:59:50.390444+0000","last_became_active":"2023-10-07T07:59:50.380510+0000","last_became_peered":"2023-10-07T07:59:50.380510+0000","last_unstale":"2023-10-07T07:59:50.390444+0000","last_undegraded":"2023-10-07T07:59:50.390444+0000","last_fullsized":"2023-10-07T07:59:50.390444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:27:54.622462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.391656+0000","last_change":"2023-10-07T07:59:49.374719+0000","last_active":"2023-10-07T07:59:50.391656+0000","last_peered":"2023-10-07T07:59:50.391656+0000","last_clean":"2023-10-07T07:59:50.391656+0000","last_became_active":"2023-10-07T07:59:49.374337+0000","last_became_peered":"2023-10-07T07:59:49.374337+0000","last_unstale":"2023-10-07T07:59:50.391656+0000","last_undegraded":"2023-10-07T07:59:50.391656+0000","last_fullsized":"2023-10-07T07:59:50.391656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:20:59.576519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.383261+0000","last_change":"2023-10-07T07:59:50.376592+0000","last_active":"2023-10-07T07:59:50.383261+0000","last_peered":"2023-10-07T07:59:50.383261+0000","last_clean":"2023-10-07T07:59:50.383261+0000","last_became_active":"2023-10-07T07:59:50.376318+0000","last_became_peered":"2023-10-07T07:59:50.376318+0000","last_unstale":"2023-10-07T07:59:50.383261+0000","last_undegraded":"2023-10-07T07:59:50.383261+0000","last_fullsized":"2023-10-07T07:59:50.383261+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:40:45.957885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.372516+0000","last_change":"2023-10-07T07:59:49.372259+0000","last_active":"2023-10-07T07:59:50.372516+0000","last_peered":"2023-10-07T07:59:50.372516+0000","last_clean":"2023-10-07T07:59:50.372516+0000","last_became_active":"2023-10-07T07:59:49.371954+0000","last_became_peered":"2023-10-07T07:59:49.371954+0000","last_unstale":"2023-10-07T07:59:50.372516+0000","last_undegraded":"2023-10-07T07:59:50.372516+0000","last_fullsized":"2023-10-07T07:59:50.372516+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:22:25.688208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.394167+0000","last_change":"2023-10-07T07:59:50.388442+0000","last_active":"2023-10-07T07:59:50.394167+0000","last_peered":"2023-10-07T07:59:50.394167+0000","last_clean":"2023-10-07T07:59:50.394167+0000","last_became_active":"2023-10-07T07:59:50.388068+0000","last_became_peered":"2023-10-07T07:59:50.388068+0000","last_unstale":"2023-10-07T07:59:50.394167+0000","last_undegraded":"2023-10-07T07:59:50.394167+0000","last_fullsized":"2023-10-07T07:59:50.394167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:01:43.243398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.373519+0000","last_change":"2023-10-07T07:59:49.374564+0000","last_active":"2023-10-07T07:59:50.373519+0000","last_peered":"2023-10-07T07:59:50.373519+0000","last_clean":"2023-10-07T07:59:50.373519+0000","last_became_active":"2023-10-07T07:59:49.374269+0000","last_became_peered":"2023-10-07T07:59:49.374269+0000","last_unstale":"2023-10-07T07:59:50.373519+0000","last_undegraded":"2023-10-07T07:59:50.373519+0000","last_fullsized":"2023-10-07T07:59:50.373519+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T11:06:12.440274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384501+0000","last_change":"2023-10-07T07:59:50.377812+0000","last_active":"2023-10-07T07:59:50.384501+0000","last_peered":"2023-10-07T07:59:50.384501+0000","last_clean":"2023-10-07T07:59:50.384501+0000","last_became_active":"2023-10-07T07:59:50.377017+0000","last_became_peered":"2023-10-07T07:59:50.377017+0000","last_unstale":"2023-10-07T07:59:50.384501+0000","last_undegraded":"2023-10-07T07:59:50.384501+0000","last_fullsized":"2023-10-07T07:59:50.384501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:27:54.622462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.369016+0000","last_change":"2023-10-07T07:59:49.376243+0000","last_active":"2023-10-07T07:59:50.369016+0000","last_peered":"2023-10-07T07:59:50.369016+0000","last_clean":"2023-10-07T07:59:50.369016+0000","last_became_active":"2023-10-07T07:59:49.375968+0000","last_became_peered":"2023-10-07T07:59:49.375968+0000","last_unstale":"2023-10-07T07:59:50.369016+0000","last_undegraded":"2023-10-07T07:59:50.369016+0000","last_fullsized":"2023-10-07T07:59:50.369016+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T15:19:57.632171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388223+0000","last_change":"2023-10-07T07:59:50.380281+0000","last_active":"2023-10-07T07:59:50.388223+0000","last_peered":"2023-10-07T07:59:50.388223+0000","last_clean":"2023-10-07T07:59:50.388223+0000","last_became_active":"2023-10-07T07:59:50.378933+0000","last_became_peered":"2023-10-07T07:59:50.378933+0000","last_unstale":"2023-10-07T07:59:50.388223+0000","last_undegraded":"2023-10-07T07:59:50.388223+0000","last_fullsized":"2023-10-07T07:59:50.388223+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:47:13.496115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.367713+0000","last_change":"2023-10-07T07:59:49.371241+0000","last_active":"2023-10-07T07:59:50.367713+0000","last_peered":"2023-10-07T07:59:50.367713+0000","last_clean":"2023-10-07T07:59:50.367713+0000","last_became_active":"2023-10-07T07:59:49.370957+0000","last_became_peered":"2023-10-07T07:59:49.370957+0000","last_unstale":"2023-10-07T07:59:50.367713+0000","last_undegraded":"2023-10-07T07:59:50.367713+0000","last_fullsized":"2023-10-07T07:59:50.367713+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T19:40:44.953040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390557+0000","last_change":"2023-10-07T07:59:50.383939+0000","last_active":"2023-10-07T07:59:50.390557+0000","last_peered":"2023-10-07T07:59:50.390557+0000","last_clean":"2023-10-07T07:59:50.390557+0000","last_became_active":"2023-10-07T07:59:50.383631+0000","last_became_peered":"2023-10-07T07:59:50.383631+0000","last_unstale":"2023-10-07T07:59:50.390557+0000","last_undegraded":"2023-10-07T07:59:50.390557+0000","last_fullsized":"2023-10-07T07:59:50.390557+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:22:45.457050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.372632+0000","last_change":"2023-10-07T07:59:49.373555+0000","last_active":"2023-10-07T07:59:50.372632+0000","last_peered":"2023-10-07T07:59:50.372632+0000","last_clean":"2023-10-07T07:59:50.372632+0000","last_became_active":"2023-10-07T07:59:49.373228+0000","last_became_peered":"2023-10-07T07:59:49.373228+0000","last_unstale":"2023-10-07T07:59:50.372632+0000","last_undegraded":"2023-10-07T07:59:50.372632+0000","last_fullsized":"2023-10-07T07:59:50.372632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:59:54.154634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.388157+0000","last_change":"2023-10-07T07:59:50.379372+0000","last_active":"2023-10-07T07:59:50.388157+0000","last_peered":"2023-10-07T07:59:50.388157+0000","last_clean":"2023-10-07T07:59:50.388157+0000","last_became_active":"2023-10-07T07:59:50.378835+0000","last_became_peered":"2023-10-07T07:59:50.378835+0000","last_unstale":"2023-10-07T07:59:50.388157+0000","last_undegraded":"2023-10-07T07:59:50.388157+0000","last_fullsized":"2023-10-07T07:59:50.388157+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:57:00.460393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.362102+0000","last_change":"2023-10-07T07:59:49.374898+0000","last_active":"2023-10-07T07:59:50.362102+0000","last_peered":"2023-10-07T07:59:50.362102+0000","last_clean":"2023-10-07T07:59:50.362102+0000","last_became_active":"2023-10-07T07:59:49.374639+0000","last_became_peered":"2023-10-07T07:59:49.374639+0000","last_unstale":"2023-10-07T07:59:50.362102+0000","last_undegraded":"2023-10-07T07:59:50.362102+0000","last_fullsized":"2023-10-07T07:59:50.362102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:16:16.654046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390145+0000","last_change":"2023-10-07T07:59:50.380829+0000","last_active":"2023-10-07T07:59:50.390145+0000","last_peered":"2023-10-07T07:59:50.390145+0000","last_clean":"2023-10-07T07:59:50.390145+0000","last_became_active":"2023-10-07T07:59:50.380110+0000","last_became_peered":"2023-10-07T07:59:50.380110+0000","last_unstale":"2023-10-07T07:59:50.390145+0000","last_undegraded":"2023-10-07T07:59:50.390145+0000","last_fullsized":"2023-10-07T07:59:50.390145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T14:40:41.701801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.373150+0000","last_change":"2023-10-07T07:59:49.374567+0000","last_active":"2023-10-07T07:59:50.373150+0000","last_peered":"2023-10-07T07:59:50.373150+0000","last_clean":"2023-10-07T07:59:50.373150+0000","last_became_active":"2023-10-07T07:59:49.374099+0000","last_became_peered":"2023-10-07T07:59:49.374099+0000","last_unstale":"2023-10-07T07:59:50.373150+0000","last_undegraded":"2023-10-07T07:59:50.373150+0000","last_fullsized":"2023-10-07T07:59:50.373150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:17:51.045372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384479+0000","last_change":"2023-10-07T07:59:50.376601+0000","last_active":"2023-10-07T07:59:50.384479+0000","last_peered":"2023-10-07T07:59:50.384479+0000","last_clean":"2023-10-07T07:59:50.384479+0000","last_became_active":"2023-10-07T07:59:50.376128+0000","last_became_peered":"2023-10-07T07:59:50.376128+0000","last_unstale":"2023-10-07T07:59:50.384479+0000","last_undegraded":"2023-10-07T07:59:50.384479+0000","last_fullsized":"2023-10-07T07:59:50.384479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T12:25:31.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.367364+0000","last_change":"2023-10-07T07:59:49.382089+0000","last_active":"2023-10-07T07:59:50.367364+0000","last_peered":"2023-10-07T07:59:50.367364+0000","last_clean":"2023-10-07T07:59:50.367364+0000","last_became_active":"2023-10-07T07:59:49.381776+0000","last_became_peered":"2023-10-07T07:59:49.381776+0000","last_unstale":"2023-10-07T07:59:50.367364+0000","last_undegraded":"2023-10-07T07:59:50.367364+0000","last_fullsized":"2023-10-07T07:59:50.367364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T17:40:47.150577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.390325+0000","last_change":"2023-10-07T07:59:50.387518+0000","last_active":"2023-10-07T07:59:50.390325+0000","last_peered":"2023-10-07T07:59:50.390325+0000","last_clean":"2023-10-07T07:59:50.390325+0000","last_became_active":"2023-10-07T07:59:50.387180+0000","last_became_peered":"2023-10-07T07:59:50.387180+0000","last_unstale":"2023-10-07T07:59:50.390325+0000","last_undegraded":"2023-10-07T07:59:50.390325+0000","last_fullsized":"2023-10-07T07:59:50.390325+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:05:30.505062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.362584+0000","last_change":"2023-10-07T07:59:49.379340+0000","last_active":"2023-10-07T07:59:50.362584+0000","last_peered":"2023-10-07T07:59:50.362584+0000","last_clean":"2023-10-07T07:59:50.362584+0000","last_became_active":"2023-10-07T07:59:49.379020+0000","last_became_peered":"2023-10-07T07:59:49.379020+0000","last_unstale":"2023-10-07T07:59:50.362584+0000","last_undegraded":"2023-10-07T07:59:50.362584+0000","last_fullsized":"2023-10-07T07:59:50.362584+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:01:01.579396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393324+0000","last_change":"2023-10-07T07:59:50.389623+0000","last_active":"2023-10-07T07:59:50.393324+0000","last_peered":"2023-10-07T07:59:50.393324+0000","last_clean":"2023-10-07T07:59:50.393324+0000","last_became_active":"2023-10-07T07:59:50.389398+0000","last_became_peered":"2023-10-07T07:59:50.389398+0000","last_unstale":"2023-10-07T07:59:50.393324+0000","last_undegraded":"2023-10-07T07:59:50.393324+0000","last_fullsized":"2023-10-07T07:59:50.393324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T18:21:00.581364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.363112+0000","last_change":"2023-10-07T07:59:49.379441+0000","last_active":"2023-10-07T07:59:50.363112+0000","last_peered":"2023-10-07T07:59:50.363112+0000","last_clean":"2023-10-07T07:59:50.363112+0000","last_became_active":"2023-10-07T07:59:49.379158+0000","last_became_peered":"2023-10-07T07:59:49.379158+0000","last_unstale":"2023-10-07T07:59:50.363112+0000","last_undegraded":"2023-10-07T07:59:50.363112+0000","last_fullsized":"2023-10-07T07:59:50.363112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T13:39:26.601574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.384987+0000","last_change":"2023-10-07T07:59:50.379732+0000","last_active":"2023-10-07T07:59:50.384987+0000","last_peered":"2023-10-07T07:59:50.384987+0000","last_clean":"2023-10-07T07:59:50.384987+0000","last_became_active":"2023-10-07T07:59:50.379454+0000","last_became_peered":"2023-10-07T07:59:50.379454+0000","last_unstale":"2023-10-07T07:59:50.384987+0000","last_undegraded":"2023-10-07T07:59:50.384987+0000","last_fullsized":"2023-10-07T07:59:50.384987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:49.351844+0000","last_clean_scrub_stamp":"2023-10-07T07:59:49.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:19:52.829382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.393017+0000","last_change":"2023-10-07T07:59:49.375635+0000","last_active":"2023-10-07T07:59:50.393017+0000","last_peered":"2023-10-07T07:59:50.393017+0000","last_clean":"2023-10-07T07:59:50.393017+0000","last_became_active":"2023-10-07T07:59:49.375348+0000","last_became_peered":"2023-10-07T07:59:49.375348+0000","last_unstale":"2023-10-07T07:59:50.393017+0000","last_undegraded":"2023-10-07T07:59:50.393017+0000","last_fullsized":"2023-10-07T07:59:50.393017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:59:48.346999+0000","last_clean_scrub_stamp":"2023-10-07T07:59:48.346999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:52:48.738067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-07T07:59:50.376223+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T07:59:50.376223+0000","last_peered":"2023-10-07T07:59:50.376223+0000","last_clean":"2023-10-07T07:59:50.376223+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T07:59:50.376223+0000","last_undegraded":"2023-10-07T07:59:50.376223+0000","last_fullsized":"2023-10-07T07:59:50.376223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297808,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074032,"statfs":{"total":96636764160,"available":96331808768,"internally_reserved":0,"allocated":532480,"data_stored":213317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.79800000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64300000000000002}]},{"osd":1,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":1.127,"15min":1.127},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.74299999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.50800000000000001}]},{"osd":2,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71099999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53500000000000003}]},{"osd":3,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.745,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67100000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.434},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67500000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55900000000000005}]},{"osd":5,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71199999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69699999999999995,"5min":1.048,"15min":1.048},"last":0.72299999999999998}]},{"osd":6,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.748,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72699999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.80300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297812,"kb_used_data":524,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074028,"statfs":{"total":96636764160,"available":96331804672,"internally_reserved":0,"allocated":536576,"data_stored":213871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70099999999999996}]},{"osd":1,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77700000000000002,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.68700000000000006,"15min":0.69799999999999995},"min":{"1min":0.39500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.76800000000000002,"15min":0.79500000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.85099999999999998,"5min":2.653,"15min":2.653},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88100000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.72499999999999998,"15min":0.747},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72099999999999997,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72399999999999998}]},{"osd":4,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.505,"15min":0.51000000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.627}]},{"osd":5,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.55500000000000005,"15min":0.56799999999999995},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73899999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":803721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.46700000000000003,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56599999999999995,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61099999999999999,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.32700000000000001}]},{"osd":2,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.754,"15min":0.80600000000000005},"min":{"1min":0.28799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.746,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.71599999999999997,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67000000000000004,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.63800000000000001,"15min":0.66200000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69599999999999995,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86499999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96899999999999997,"5min":1.452,"15min":1.452},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.82999999999999996,"15min":0.85299999999999998},"min":{"1min":0.51900000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90800000000000003,"5min":3.3929999999999998,"15min":3.3929999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0029999999999999,"5min":1.546,"15min":1.546},"last":0.74199999999999999}]},{"osd":6,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.58399999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0209999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.86499999999999999,"15min":0.88700000000000001},"min":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0569999999999999,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Oct 7 07:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.874,"15min":0.89600000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.94299999999999995,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0369999999999999,"5min":1.585,"15min":1.585},"last":0.70899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297812,"kb_used_data":524,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074028,"statfs":{"total":96636764160,"available":96331804672,"internally_reserved":0,"allocated":536576,"data_stored":213927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.501,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56699999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.438,"15min":0.443},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58499999999999996,"5min":1.298,"15min":1.298},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":1.073,"15min":1.073},"last":0.628}]},{"osd":3,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.63100000000000001,"5min":1.032,"15min":1.032},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.35499999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59699999999999998,"5min":1.141,"15min":1.141},"last":0.64000000000000001}]},{"osd":4,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69599999999999995,"15min":0.70099999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.81299999999999994,"5min":1.218,"15min":1.218},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.52900000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86699999999999999,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.91300000000000003}]},{"osd":5,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.876,"5min":1.375,"15min":1.375},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82799999999999996,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88800000000000001,"5min":1.401,"15min":1.401},"last":0.872},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90400000000000003,"5min":1.462,"15min":1.462},"last":0.90200000000000002}]},{"osd":7,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74399999999999999,"15min":0.746},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.86199999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.80100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297808,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074032,"statfs":{"total":96636764160,"available":96331808768,"internally_reserved":0,"allocated":532480,"data_stored":213373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.504,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40100000000000002,"15min":0.39900000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.31900000000000001}]},{"osd":1,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.377},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.49299999999999999,"5min":0.747,"15min":0.747},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.38300000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34100000000000003}]},{"osd":4,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64300000000000002,"15min":0.63800000000000001},"min":{"1min":0.56899999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.80300000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.54800000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83499999999999996,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.755}]},{"osd":5,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.49299999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.81200000000000006,"5min":1.51,"15min":1.51},"last":0.80200000000000005}]},{"osd":6,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.51000000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.67800000000000005}]},{"osd":7,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.504,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84199999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85599999999999998,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.81799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":540672,"data_stored":214819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.26900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.433,"15min":0.437},"min":{"1min":0.317,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.5,"5min":1.016,"15min":1.016},"last":0.64100000000000001}]},{"osd":1,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.46000000000000002,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.55600000000000005,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.502},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52500000000000002,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.57699999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.443}]},{"osd":2,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.51300000000000001,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.60399999999999998,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.61899999999999999,"5min":1.323,"15min":1.323},"last":0.59699999999999998}]},{"osd":4,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.52600000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.79000000000000004,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83899999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.84099999999999997,"5min":1.327,"15min":1.327},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.82899999999999996,"15min":0.85199999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.80900000000000005,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.79700000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.86099999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.81399999999999995,"15min":0.83099999999999996},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.82999999999999996,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.018}]},{"osd":7,"last update":"Sat Oct 7 07:59:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.81599999999999995,"5min":1.528,"15min":1.528},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.82399999999999995,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0369999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":214911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39900000000000002}]},{"osd":2,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46999999999999997}]},{"osd":3,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.749,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55600000000000005}]},{"osd":5,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.318,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65000000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72399999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.70599999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.40699999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73399999999999999,"5min":0.878,"15min":0.878},"last":0.62}]},{"osd":7,"last update":"Sat Oct 7 07:59:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64000000000000001,"15min":0.64700000000000002},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":803673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72299999999999998,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.70799999999999996,"15min":0.72499999999999998},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.85199999999999998,"5min":3.375,"15min":3.375},"last":0.67900000000000005}]},{"osd":1,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.36499999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73399999999999999,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.67000000000000004,"15min":0.68500000000000005},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.748,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.748,"15min":0.77200000000000002},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80200000000000005,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.88},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.78900000000000003,"15min":0.82399999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80900000000000005,"5min":5.2699999999999996,"15min":5.2699999999999996},"last":0.79100000000000004}]},{"osd":3,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.74299999999999999,"15min":0.77100000000000002},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79900000000000004,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.0569999999999999,"15min":1.133},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73399999999999999,"5min":10.644,"15min":10.644},"last":0.84299999999999997}]},{"osd":4,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72799999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":1.242,"15min":1.242},"last":0.54300000000000004}]},{"osd":6,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78300000000000003,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Sat Oct 7 07:59:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79100000000000004,"5min":1.123,"15min":1.123},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T07:59:54.980 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-07T07:59:54.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-07T07:59:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.153+0000 7f8bc7fdb700 1 Processor -- start 2023-10-07T07:59:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.153+0000 7f8bc7fdb700 1 -- start start 2023-10-07T07:59:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc7fdb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8bc0147030 0x7f8bc015e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc7fdb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8bc006e3c0 0x7f8bc0066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc7fdb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147400 0x7f8bc0067530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc7fdb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc006a500 con 0x7f8bc0147030 2023-10-07T07:59:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc7fdb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc00cccb0 con 0x7f8bc006e3c0 2023-10-07T07:59:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc7fdb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00cae60 con 0x7f8bc0147400 2023-10-07T07:59:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5d77700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8bc0147030 0x7f8bc015e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5d77700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8bc0147030 0x7f8bc015e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38582/0 (socket says 172.21.15.7:38582) 2023-10-07T07:59:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5d77700 1 -- 172.21.15.7:0/1678314415 learned_addr learned my addr 172.21.15.7:0/1678314415 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc6578700 1 --2- 172.21.15.7:0/1678314415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147400 0x7f8bc0067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5576700 1 --2- 172.21.15.7:0/1678314415 >> v2:172.21.15.172:3300/0 conn(0x7f8bc006e3c0 0x7f8bc0066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5d77700 1 -- 172.21.15.7:0/1678314415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147400 msgr2=0x7f8bc0067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5d77700 1 --2- 172.21.15.7:0/1678314415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147400 0x7f8bc0067530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5d77700 1 -- 172.21.15.7:0/1678314415 >> v2:172.21.15.172:3300/0 conn(0x7f8bc006e3c0 msgr2=0x7f8bc0066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5d77700 1 --2- 172.21.15.7:0/1678314415 >> v2:172.21.15.172:3300/0 conn(0x7f8bc006e3c0 0x7f8bc0066ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5d77700 1 -- 172.21.15.7:0/1678314415 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc015eca0 con 0x7f8bc0147030 2023-10-07T07:59:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.154+0000 7f8bc5576700 1 --2- 172.21.15.7:0/1678314415 >> v2:172.21.15.172:3300/0 conn(0x7f8bc006e3c0 0x7f8bc0066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.155+0000 7f8bc5d77700 1 --2- 172.21.15.7:0/1678314415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147030 0x7f8bc015e760 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8bc006e2c0 tx=0x7f8bb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d99d4ef86a3f38 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.155+0000 7f8bc4d75700 1 -- 172.21.15.7:0/1678314415 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4009e70 con 0x7f8bc0147030 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.155+0000 7f8bc4d75700 1 -- 172.21.15.7:0/1678314415 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb400b860 con 0x7f8bc0147030 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.155+0000 7f8bc4d75700 1 -- 172.21.15.7:0/1678314415 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400bc00 con 0x7f8bc0147030 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.155+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1678314415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147030 msgr2=0x7f8bc015e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.155+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1678314415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147030 0x7f8bc015e760 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8bc006e2c0 tx=0x7f8bb4009250 comp rx=0 tx=0).stop 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.156+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1678314415 shutdown_connections 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.156+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1678314415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147400 0x7f8bc0067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.156+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1678314415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147030 0x7f8bc015e760 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.156+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1678314415 >> v2:172.21.15.172:3300/0 conn(0x7f8bc006e3c0 0x7f8bc0066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.156+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1678314415 >> 172.21.15.7:0/1678314415 conn(0x7f8bc0068100 msgr2=0x7f8bc0068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.156+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1678314415 shutdown_connections 2023-10-07T07:59:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.156+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1678314415 wait complete. 2023-10-07T07:59:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.157+0000 7f8bc7fdb700 1 Processor -- start 2023-10-07T07:59:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.157+0000 7f8bc7fdb700 1 -- start start 2023-10-07T07:59:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc7fdb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc006e3c0 0x7f8bc015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc7fdb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147030 0x7f8bc0161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc7fdb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147400 0x7f8bc00ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc7fdb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00ad9b0 con 0x7f8bc0147400 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc7fdb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc006ab80 con 0x7f8bc006e3c0 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc7fdb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00ad110 con 0x7f8bc0147030 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc6578700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147400 0x7f8bc00ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc6578700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147400 0x7f8bc00ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38594/0 (socket says 172.21.15.7:38594) 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc6578700 1 -- 172.21.15.7:0/1673819756 learned_addr learned my addr 172.21.15.7:0/1673819756 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc5d77700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc006e3c0 0x7f8bc015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc5576700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147030 0x7f8bc0161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc6578700 1 -- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147030 msgr2=0x7f8bc0161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc6578700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147030 0x7f8bc0161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc6578700 1 -- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc006e3c0 msgr2=0x7f8bc015cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc6578700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc006e3c0 0x7f8bc015cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc6578700 1 -- 172.21.15.7:0/1673819756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40094f0 con 0x7f8bc0147400 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc5576700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147030 0x7f8bc0161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc6578700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147400 0x7f8bc00ade50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009df0 tx=0x7f8bb800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/1673819756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800cb50 con 0x7f8bc0147400 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/1673819756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb800f040 con 0x7f8bc0147400 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/1673819756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8014480 con 0x7f8bc0147400 2023-10-07T07:59:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.158+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc00ad3f0 con 0x7f8bc0147400 2023-10-07T07:59:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.159+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc006a810 con 0x7f8bc0147400 2023-10-07T07:59:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.160+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc006a670 con 0x7f8bc0147400 2023-10-07T07:59:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.160+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/1673819756 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8bb8014830 con 0x7f8bc0147400 2023-10-07T07:59:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.161+0000 7f8bb1ffb700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8ba4066f30 0x7f8ba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.161+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/1673819756 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8bb8084eb0 con 0x7f8bc0147400 2023-10-07T07:59:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.161+0000 7f8bc5d77700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8ba4066f30 0x7f8ba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.162+0000 7f8bc5d77700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8ba4066f30 0x7f8ba40693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8bc006e2c0 tx=0x7f8bb4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.165+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/1673819756 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bc006a670 con 0x7f8bc0147400 2023-10-07T07:59:55.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.452+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8bc006a670 con 0x7f8bc0147400 2023-10-07T07:59:55.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.455+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/1673819756 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8bc006a670 con 0x7f8bc0147400 2023-10-07T07:59:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8ba4066f30 msgr2=0x7f8ba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8ba4066f30 0x7f8ba40693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8bc006e2c0 tx=0x7f8bb4017000 comp rx=0 tx=0).stop 2023-10-07T07:59:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147400 msgr2=0x7f8bc00ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147400 0x7f8bc00ade50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009df0 tx=0x7f8bb800b040 comp rx=0 tx=0).stop 2023-10-07T07:59:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 shutdown_connections 2023-10-07T07:59:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8ba4066f30 0x7f8ba40693e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc0147400 0x7f8bc00ade50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc006e3c0 0x7f8bc015cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 --2- 172.21.15.7:0/1673819756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc0147030 0x7f8bc0161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 >> 172.21.15.7:0/1673819756 conn(0x7f8bc0068100 msgr2=0x7f8bc0144cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:55.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 shutdown_connections 2023-10-07T07:59:55.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.458+0000 7f8bc7fdb700 1 -- 172.21.15.7:0/1673819756 wait complete. 2023-10-07T07:59:55.460 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2023-10-07T07:59:55.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T07:59:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.604+0000 7f4ecb66a700 1 Processor -- start 2023-10-07T07:59:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.604+0000 7f4ecb66a700 1 -- start start 2023-10-07T07:59:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.605+0000 7f4ecb66a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ec4063230 0x7f4ec406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.605+0000 7f4ecb66a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9ec0 0x7f4ec406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.605+0000 7f4ecb66a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec406e590 con 0x7f4ec40c9af0 2023-10-07T07:59:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.605+0000 7f4ecb66a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec40b03a0 con 0x7f4ec4063230 2023-10-07T07:59:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.605+0000 7f4ecb66a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec415a330 con 0x7f4ec40c9ec0 2023-10-07T07:59:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.605+0000 7f4ec8c05700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ec4063230 0x7f4ec406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.605+0000 7f4ec8c05700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ec4063230 0x7f4ec406db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38600/0 (socket says 172.21.15.7:38600) 2023-10-07T07:59:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.605+0000 7f4ec8c05700 1 -- 172.21.15.7:0/4194392116 learned_addr learned my addr 172.21.15.7:0/4194392116 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.605+0000 7f4ec9c07700 1 --2- 172.21.15.7:0/4194392116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9ec0 0x7f4ec406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.606+0000 7f4ec8c05700 1 -- 172.21.15.7:0/4194392116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9ec0 msgr2=0x7f4ec406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.606+0000 7f4ec8c05700 1 --2- 172.21.15.7:0/4194392116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9ec0 0x7f4ec406e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.606+0000 7f4ec8c05700 1 -- 172.21.15.7:0/4194392116 >> v1:172.21.15.172:6789/0 conn(0x7f4ec40c9af0 legacy=0x7f4ec415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.606+0000 7f4ec8c05700 1 -- 172.21.15.7:0/4194392116 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec415a610 con 0x7f4ec4063230 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.606+0000 7f4ec8c05700 1 --2- 172.21.15.7:0/4194392116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 0x7f4ec406db10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40099b0 tx=0x7f4eb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddec1407f4815670 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.606+0000 7f4ebbfff700 1 -- 172.21.15.7:0/4194392116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4009ea0 con 0x7f4ec4063230 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.606+0000 7f4ebbfff700 1 -- 172.21.15.7:0/4194392116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb400b860 con 0x7f4ec4063230 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.606+0000 7f4ebbfff700 1 -- 172.21.15.7:0/4194392116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb400bc00 con 0x7f4ec4063230 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.607+0000 7f4ecb66a700 1 -- 172.21.15.7:0/4194392116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 msgr2=0x7f4ec406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.607+0000 7f4ecb66a700 1 --2- 172.21.15.7:0/4194392116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 0x7f4ec406db10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40099b0 tx=0x7f4eb4009250 comp rx=0 tx=0).stop 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.607+0000 7f4ecb66a700 1 -- 172.21.15.7:0/4194392116 shutdown_connections 2023-10-07T07:59:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.607+0000 7f4ecb66a700 1 --2- 172.21.15.7:0/4194392116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9ec0 0x7f4ec406e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.607+0000 7f4ecb66a700 1 --2- 172.21.15.7:0/4194392116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 0x7f4ec406db10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.607+0000 7f4ecb66a700 1 -- 172.21.15.7:0/4194392116 >> 172.21.15.7:0/4194392116 conn(0x7f4ec40cd1a0 msgr2=0x7f4ec40c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:55.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.607+0000 7f4ecb66a700 1 -- 172.21.15.7:0/4194392116 shutdown_connections 2023-10-07T07:59:55.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.607+0000 7f4ecb66a700 1 -- 172.21.15.7:0/4194392116 wait complete. 2023-10-07T07:59:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.608+0000 7f4ecb66a700 1 Processor -- start 2023-10-07T07:59:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.608+0000 7f4ecb66a700 1 -- start start 2023-10-07T07:59:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ecb66a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 0x7f4ec4169920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ecb66a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9af0 0x7f4ec414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ecb66a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec40c9ec0 0x7f4ec40c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ecb66a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40c8110 con 0x7f4ec4063230 2023-10-07T07:59:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ecb66a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec415b650 con 0x7f4ec40c9ec0 2023-10-07T07:59:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ecb66a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec415ad20 con 0x7f4ec40c9af0 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec9406700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 0x7f4ec4169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec9406700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 0x7f4ec4169920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38614/0 (socket says 172.21.15.7:38614) 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec9406700 1 -- 172.21.15.7:0/285190196 learned_addr learned my addr 172.21.15.7:0/285190196 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec8c05700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9af0 0x7f4ec414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec9c07700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec40c9ec0 0x7f4ec40c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec9406700 1 -- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9af0 msgr2=0x7f4ec414efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec9406700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9af0 0x7f4ec414efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec9406700 1 -- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec40c9ec0 msgr2=0x7f4ec40c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec9406700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec40c9ec0 0x7f4ec40c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.609+0000 7f4ec9406700 1 -- 172.21.15.7:0/285190196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40094f0 con 0x7f4ec4063230 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.610+0000 7f4ec9406700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 0x7f4ec4169920 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000c790 tx=0x7f4ec0007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.610+0000 7f4eb97fa700 1 -- 172.21.15.7:0/285190196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0011590 con 0x7f4ec4063230 2023-10-07T07:59:55.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.610+0000 7f4eb97fa700 1 -- 172.21.15.7:0/285190196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec00116f0 con 0x7f4ec4063230 2023-10-07T07:59:55.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.610+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec415b000 con 0x7f4ec4063230 2023-10-07T07:59:55.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.610+0000 7f4eb97fa700 1 -- 172.21.15.7:0/285190196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0011ab0 con 0x7f4ec4063230 2023-10-07T07:59:55.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.610+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec414f510 con 0x7f4ec4063230 2023-10-07T07:59:55.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.611+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec4150a00 con 0x7f4ec4063230 2023-10-07T07:59:55.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.611+0000 7f4eb97fa700 1 -- 172.21.15.7:0/285190196 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ec001b3b0 con 0x7f4ec4063230 2023-10-07T07:59:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.612+0000 7f4eb97fa700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4ea8066f30 0x7f4ea80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.612+0000 7f4eb97fa700 1 -- 172.21.15.7:0/285190196 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ec00839e0 con 0x7f4ec4063230 2023-10-07T07:59:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.612+0000 7f4ec8c05700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4ea8066f30 0x7f4ea80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.612+0000 7f4ec8c05700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4ea8066f30 0x7f4ea80693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40099b0 tx=0x7f4eb4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:55.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.616+0000 7f4eb97fa700 1 -- 172.21.15.7:0/285190196 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ec0050c50 con 0x7f4ec4063230 2023-10-07T07:59:55.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.866+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ec40716c0 con 0x7f4ec4063230 2023-10-07T07:59:55.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.867+0000 7f4eb97fa700 1 -- 172.21.15.7:0/285190196 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4588 (secure 0 0 0) 0x7f4ec0017070 con 0x7f4ec4063230 2023-10-07T07:59:55.868 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T07:59:55.868 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3622385241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3622385241},{"type":"v1","addr":"172.21.15.172:6837","nonce":3622385241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/899764672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":899764672},{"type":"v1","addr":"172.21.15.172:6836","nonce":899764672}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/318366060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":318366060},{"type":"v1","addr":"172.21.15.7:6833","nonce":318366060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:59:49.797329+0000","modified":"2023-10-07T07:59:53.390983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/763397529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":763397529},{"type":"v1","addr":"172.21.15.7:6835","nonce":763397529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T07:59:55.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4ea8066f30 msgr2=0x7f4ea80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4ea8066f30 0x7f4ea80693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40099b0 tx=0x7f4eb4017000 comp rx=0 tx=0).stop 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 msgr2=0x7f4ec4169920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 0x7f4ec4169920 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000c790 tx=0x7f4ec0007f40 comp rx=0 tx=0).stop 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 shutdown_connections 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4ea8066f30 0x7f4ea80693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec40c9ec0 0x7f4ec40c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec4063230 0x7f4ec4169920 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 --2- 172.21.15.7:0/285190196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ec40c9af0 0x7f4ec414efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 >> 172.21.15.7:0/285190196 conn(0x7f4ec40cd1a0 msgr2=0x7f4ec41502f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 shutdown_connections 2023-10-07T07:59:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:55.869+0000 7f4ecb66a700 1 -- 172.21.15.7:0/285190196 wait complete. 2023-10-07T07:59:55.872 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2023-10-07T07:59:56.287 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-07T07:59:49.797329+0000', 'modified': '2023-10-07T07:59:53.390983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5382}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5382': {'gid': 5382, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/763397529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 763397529}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 763397529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-07T07:59:56.287 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-07T07:59:56.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-07T07:59:56.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.429+0000 7f8ddc305700 1 Processor -- start 2023-10-07T07:59:56.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.429+0000 7f8ddc305700 1 -- start start 2023-10-07T07:59:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.430+0000 7f8ddc305700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 0x7f8dd406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.430+0000 7f8ddc305700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8dd4159690 0x7f8dd406a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.430+0000 7f8ddc305700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd40cab80 0x7f8dd40cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.430+0000 7f8ddc305700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd406bb70 con 0x7f8dd40cab80 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.430+0000 7f8ddc305700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd40cda10 con 0x7f8dd4159690 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.430+0000 7f8ddc305700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd406af60 con 0x7f8dd41592c0 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda8a2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd40cab80 0x7f8dd40cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda8a2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dd40cab80 0x7f8dd40cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38618/0 (socket says 172.21.15.7:38618) 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda8a2700 1 -- 172.21.15.7:0/405741620 learned_addr learned my addr 172.21.15.7:0/405741620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/405741620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 0x7f8dd406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dd98a0700 1 --2- 172.21.15.7:0/405741620 >> v2:172.21.15.172:3300/0 conn(0x7f8dd4159690 0x7f8dd406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda8a2700 1 -- 172.21.15.7:0/405741620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 msgr2=0x7f8dd406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/405741620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 0x7f8dd406a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda8a2700 1 -- 172.21.15.7:0/405741620 >> v2:172.21.15.172:3300/0 conn(0x7f8dd4159690 msgr2=0x7f8dd406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/405741620 >> v2:172.21.15.172:3300/0 conn(0x7f8dd4159690 0x7f8dd406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda8a2700 1 -- 172.21.15.7:0/405741620 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd406b240 con 0x7f8dd40cab80 2023-10-07T07:59:56.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/405741620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 0x7f8dd40cafe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8dd406e0b0 tx=0x7f8dcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50b32096fda09bb server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dd909f700 1 -- 172.21.15.7:0/405741620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc00e070 con 0x7f8dd40cab80 2023-10-07T07:59:56.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dd909f700 1 -- 172.21.15.7:0/405741620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc00b970 con 0x7f8dd40cab80 2023-10-07T07:59:56.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.431+0000 7f8dd909f700 1 -- 172.21.15.7:0/405741620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0092e0 con 0x7f8dd40cab80 2023-10-07T07:59:56.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.432+0000 7f8ddc305700 1 -- 172.21.15.7:0/405741620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 msgr2=0x7f8dd40cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.432+0000 7f8ddc305700 1 --2- 172.21.15.7:0/405741620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 0x7f8dd40cafe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8dd406e0b0 tx=0x7f8dcc009840 comp rx=0 tx=0).stop 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.432+0000 7f8ddc305700 1 -- 172.21.15.7:0/405741620 shutdown_connections 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.432+0000 7f8ddc305700 1 --2- 172.21.15.7:0/405741620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 0x7f8dd40cafe0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.432+0000 7f8ddc305700 1 --2- 172.21.15.7:0/405741620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 0x7f8dd406a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.432+0000 7f8ddc305700 1 --2- 172.21.15.7:0/405741620 >> v2:172.21.15.172:3300/0 conn(0x7f8dd4159690 0x7f8dd406a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.432+0000 7f8ddc305700 1 -- 172.21.15.7:0/405741620 >> 172.21.15.7:0/405741620 conn(0x7f8dd4068bd0 msgr2=0x7f8dd40ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.432+0000 7f8ddc305700 1 -- 172.21.15.7:0/405741620 shutdown_connections 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.432+0000 7f8ddc305700 1 -- 172.21.15.7:0/405741620 wait complete. 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.433+0000 7f8ddc305700 1 Processor -- start 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.433+0000 7f8ddc305700 1 -- start start 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.434+0000 7f8ddc305700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 0x7f8dd41568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.434+0000 7f8ddc305700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 0x7f8dd415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.434+0000 7f8ddc305700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4159690 0x7f8dd415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.434+0000 7f8ddc305700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40afd50 con 0x7f8dd40cab80 2023-10-07T07:59:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.434+0000 7f8ddc305700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40b0030 con 0x7f8dd4159690 2023-10-07T07:59:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.434+0000 7f8ddc305700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40b0310 con 0x7f8dd41592c0 2023-10-07T07:59:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.434+0000 7f8dda0a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 0x7f8dd41568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.434+0000 7f8dda0a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 0x7f8dd41568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38630/0 (socket says 172.21.15.7:38630) 2023-10-07T07:59:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.434+0000 7f8dda0a1700 1 -- 172.21.15.7:0/4133913921 learned_addr learned my addr 172.21.15.7:0/4133913921 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dd98a0700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 0x7f8dd415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4159690 0x7f8dd415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dda0a1700 1 -- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 msgr2=0x7f8dd415ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 0x7f8dd415ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dda0a1700 1 -- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4159690 msgr2=0x7f8dd415b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4159690 0x7f8dd415b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dda0a1700 1 -- 172.21.15.7:0/4133913921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc009ae0 con 0x7f8dd40cab80 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4159690 0x7f8dd415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 0x7f8dd41568c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800ec80 tx=0x7f8dc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dc67fc700 1 -- 172.21.15.7:0/4133913921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800c9d0 con 0x7f8dd40cab80 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dc67fc700 1 -- 172.21.15.7:0/4133913921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc800cb30 con 0x7f8dd40cab80 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8ddc305700 1 -- 172.21.15.7:0/4133913921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd40b05f0 con 0x7f8dd40cab80 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8dc67fc700 1 -- 172.21.15.7:0/4133913921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8013470 con 0x7f8dd40cab80 2023-10-07T07:59:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.435+0000 7f8ddc305700 1 -- 172.21.15.7:0/4133913921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd40b0bb0 con 0x7f8dd40cab80 2023-10-07T07:59:56.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.436+0000 7f8dc67fc700 1 -- 172.21.15.7:0/4133913921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8dc8013820 con 0x7f8dd40cab80 2023-10-07T07:59:56.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.437+0000 7f8dc67fc700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8db8066f30 0x7f8db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.437+0000 7f8dc67fc700 1 -- 172.21.15.7:0/4133913921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8dc80848a0 con 0x7f8dd40cab80 2023-10-07T07:59:56.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.437+0000 7f8dd98a0700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8db8066f30 0x7f8db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.438+0000 7f8dd98a0700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8db8066f30 0x7f8db80693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40c8720 tx=0x7f8dd0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.439+0000 7f8de5619b80 1 Processor -- start 2023-10-07T07:59:56.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.439+0000 7f8de5619b80 1 -- start start 2023-10-07T07:59:56.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 0x55dda92f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92d8530 0x55dda92fb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8de5619b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda91c5310 con 0x55dda92fbbc0 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8de5619b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda9202130 con 0x55dda92d8530 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8de5619b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dda91ffaf0 con 0x55dda92f2a40 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8dda8a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8dda8a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38632/0 (socket says 172.21.15.7:38632) 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8dda8a2700 1 -- 172.21.15.7:0/2717287525 learned_addr learned my addr 172.21.15.7:0/2717287525 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/2717287525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 0x55dda92f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.440+0000 7f8dd98a0700 1 --2- 172.21.15.7:0/2717287525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92d8530 0x55dda92fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.441+0000 7f8dda8a2700 1 -- 172.21.15.7:0/2717287525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 msgr2=0x55dda92f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.441+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/2717287525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 0x55dda92f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.441+0000 7f8dda8a2700 1 -- 172.21.15.7:0/2717287525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92d8530 msgr2=0x55dda92fb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.441+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/2717287525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92d8530 0x55dda92fb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.441+0000 7f8dda8a2700 1 -- 172.21.15.7:0/2717287525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dda90ee0a0 con 0x55dda92fbbc0 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.441+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/2717287525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92fc010 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc012000 tx=0x7f8dcc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d2e31c052e4643 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.441+0000 7f8db77fe700 1 -- 172.21.15.7:0/2717287525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0032f0 con 0x55dda92fbbc0 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.441+0000 7f8db77fe700 1 -- 172.21.15.7:0/2717287525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc00bae0 con 0x55dda92fbbc0 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.441+0000 7f8db77fe700 1 -- 172.21.15.7:0/2717287525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc01d670 con 0x55dda92fbbc0 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 -- 172.21.15.7:0/2717287525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 msgr2=0x55dda92fc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 --2- 172.21.15.7:0/2717287525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92fc010 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc012000 tx=0x7f8dcc018000 comp rx=0 tx=0).stop 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 -- 172.21.15.7:0/2717287525 shutdown_connections 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 --2- 172.21.15.7:0/2717287525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92fc010 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 --2- 172.21.15.7:0/2717287525 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 0x55dda92f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 --2- 172.21.15.7:0/2717287525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92d8530 0x55dda92fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 -- 172.21.15.7:0/2717287525 >> 172.21.15.7:0/2717287525 conn(0x55dda92f5e70 msgr2=0x55dda92f6240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 -- 172.21.15.7:0/2717287525 shutdown_connections 2023-10-07T07:59:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 -- 172.21.15.7:0/2717287525 wait complete. 2023-10-07T07:59:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.442+0000 7f8de5619b80 1 -- start start 2023-10-07T07:59:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.443+0000 7f8de5619b80 1 Processor -- start 2023-10-07T07:59:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 0x55dda92d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 0x55dda92d43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8de5619b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda91fb8e0 con 0x55dda92fbbc0 2023-10-07T07:59:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8de5619b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda91a7980 con 0x55dda92f2a40 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8de5619b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dda91a7620 con 0x55dda92d8530 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8de5619b80 10 client.0 _ll_register_callbacks cb 0x55dda900aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda8a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda8a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38634/0 (socket says 172.21.15.7:38634) 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda8a2700 1 -- 172.21.15.7:0/1052459343 learned_addr learned my addr 172.21.15.7:0/1052459343 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dd98a0700 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 0x55dda92d43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 0x55dda92d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda8a2700 1 -- 172.21.15.7:0/1052459343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 msgr2=0x55dda92d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 0x55dda92d89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda8a2700 1 -- 172.21.15.7:0/1052459343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 msgr2=0x55dda92d43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 0x55dda92d43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda8a2700 1 -- 172.21.15.7:0/1052459343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd406b240 con 0x55dda92fbbc0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d1ad0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00b610 tx=0x7f8dcc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb6beecc687380f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8db4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.444+0000 7f8db4ff9700 1 -- 172.21.15.7:0/1052459343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc02b070 con 0x55dda92fbbc0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8db4ff9700 1 -- 172.21.15.7:0/1052459343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8dcc01dc30 con 0x55dda92fbbc0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8de5619b80 1 -- 172.21.15.7:0/1052459343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dda92a0560 con 0x55dda92fbbc0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8db4ff9700 1 -- 172.21.15.7:0/1052459343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dcc01de10 con 0x55dda92fbbc0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8de5619b80 10 client.5517 fetch_fsmap learned FSMap version 25 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8de5619b80 1 -- 172.21.15.7:0/1052459343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55dda91ca3c0 con 0x55dda92fbbc0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8db4ff9700 1 -- 172.21.15.7:0/1052459343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dda92a0560 con 0x55dda92fbbc0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8db4ff9700 1 -- 172.21.15.7:0/1052459343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8dcc019070 con 0x55dda92fbbc0 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8de5619b80 10 client.5517 fetch_fsmap finished waiting for FSMap version 25 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8de5619b80 10 client.5517 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-07T07:59:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8de5619b80 20 client.5517 populate_metadata read hostname 'smithi007' 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8de5619b80 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] conn(0x55dda92d7770 0x55dda92cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8de5619b80 4 client.5517 mds_command: new command op to 5382 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8de5619b80 1 -- 172.21.15.7:0/1052459343 --> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dda927d0e0 con 0x55dda92d7770 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.445+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] conn(0x55dda92d7770 0x55dda92cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8db4ff9700 10 client.5517 ms_handle_connect on v2:172.21.15.7:6834/763397529 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] conn(0x55dda92d7770 0x55dda92cf920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71406295af10bc20 server_cookie=74e6079380606b4a in_seq=0 out_seq=0 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8db4ff9700 1 -- 172.21.15.7:0/1052459343 <== mds.0 v2:172.21.15.7:6834/763397529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55dda927d0e0 con 0x55dda92d7770 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8db4ff9700 10 client.5517 handle_command_reply: tid=0 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8de5619b80 1 client.5517 shutdown 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8de5619b80 1 -- 172.21.15.7:0/1052459343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 msgr2=0x55dda92d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8de5619b80 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d1ad0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00b610 tx=0x7f8dcc01e000 comp rx=0 tx=0).stop 2023-10-07T07:59:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8de5619b80 1 -- 172.21.15.7:0/1052459343 shutdown_connections 2023-10-07T07:59:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8de5619b80 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] conn(0x55dda92d7770 0x55dda92cf920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8de5619b80 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d1ad0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8de5619b80 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 0x55dda92d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8de5619b80 1 --2- 172.21.15.7:0/1052459343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 0x55dda92d43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8db4ff9700 0 client.5517 ms_handle_reset on v2:172.21.15.7:6834/763397529 2023-10-07T07:59:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.446+0000 7f8de5619b80 1 -- 172.21.15.7:0/1052459343 >> 172.21.15.7:0/1052459343 conn(0x55dda92f5e70 msgr2=0x55dda92a8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.447+0000 7f8de5619b80 1 -- 172.21.15.7:0/1052459343 shutdown_connections 2023-10-07T07:59:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.447+0000 7f8de5619b80 1 -- 172.21.15.7:0/1052459343 wait complete. 2023-10-07T07:59:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.447+0000 7f8de5619b80 20 client.5517 trim_cache size 0 max 16384 2023-10-07T07:59:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.465+0000 7f8de5619b80 1 Processor -- start 2023-10-07T07:59:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.465+0000 7f8de5619b80 1 -- start start 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92d8530 0x55dda92d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 0x55dda92a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92fbbc0 0x55dda92ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8de5619b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda90931e0 con 0x55dda92d8530 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8de5619b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda9348e70 con 0x55dda92fbbc0 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8de5619b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dda92d8bc0 con 0x55dda92f2a40 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda0a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92d8530 0x55dda92d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda0a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92d8530 0x55dda92d3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38642/0 (socket says 172.21.15.7:38642) 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda0a1700 1 -- 172.21.15.7:0/2596485259 learned_addr learned my addr 172.21.15.7:0/2596485259 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dd98a0700 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 0x55dda92a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92fbbc0 0x55dda92ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda0a1700 1 -- 172.21.15.7:0/2596485259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 msgr2=0x55dda92a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 0x55dda92a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda0a1700 1 -- 172.21.15.7:0/2596485259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92fbbc0 msgr2=0x55dda92ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92fbbc0 0x55dda92ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda0a1700 1 -- 172.21.15.7:0/2596485259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dda90621d0 con 0x55dda92d8530 2023-10-07T07:59:56.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dd98a0700 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 0x55dda92a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T07:59:56.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92d8530 0x55dda92d3770 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8055710 tx=0x7f8dc8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169e9ce7b4246beb server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8db7fff700 1 -- 172.21.15.7:0/2596485259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc805c770 con 0x55dda92d8530 2023-10-07T07:59:56.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8db7fff700 1 -- 172.21.15.7:0/2596485259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc8082ed0 con 0x55dda92d8530 2023-10-07T07:59:56.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.466+0000 7f8db7fff700 1 -- 172.21.15.7:0/2596485259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8053910 con 0x55dda92d8530 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.467+0000 7f8de5619b80 1 -- 172.21.15.7:0/2596485259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92d8530 msgr2=0x55dda92d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.467+0000 7f8de5619b80 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92d8530 0x55dda92d3770 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8055710 tx=0x7f8dc8015040 comp rx=0 tx=0).stop 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.467+0000 7f8de5619b80 1 -- 172.21.15.7:0/2596485259 shutdown_connections 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.467+0000 7f8de5619b80 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92fbbc0 0x55dda92ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.467+0000 7f8de5619b80 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92d8530 0x55dda92d3770 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.467+0000 7f8de5619b80 1 --2- 172.21.15.7:0/2596485259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92f2a40 0x55dda92a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.467+0000 7f8de5619b80 1 -- 172.21.15.7:0/2596485259 >> 172.21.15.7:0/2596485259 conn(0x55dda92f5e70 msgr2=0x55dda92d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.467+0000 7f8de5619b80 1 -- 172.21.15.7:0/2596485259 shutdown_connections 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.467+0000 7f8de5619b80 1 -- 172.21.15.7:0/2596485259 wait complete. 2023-10-07T07:59:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.468+0000 7f8de5619b80 1 -- start start 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.468+0000 7f8de5619b80 1 Processor -- start 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 0x55dda9347a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 0x55dda93579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8de5619b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8de5619b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda92d53c0 con 0x55dda92fbbc0 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8de5619b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda9357ee0 con 0x55dda92f2a40 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8de5619b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dda9358080 con 0x55dda92d8530 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8de5619b80 10 client.0 _ll_register_callbacks cb 0x55dda900aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dda8a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dda8a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38658/0 (socket says 172.21.15.7:38658) 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dda8a2700 1 -- 172.21.15.7:0/477272371 learned_addr learned my addr 172.21.15.7:0/477272371 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 0x55dda9347a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dd98a0700 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 0x55dda93579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dda8a2700 1 -- 172.21.15.7:0/477272371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 msgr2=0x55dda9347a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 0x55dda9347a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dda8a2700 1 -- 172.21.15.7:0/477272371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 msgr2=0x55dda93579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 0x55dda93579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.469+0000 7f8dda8a2700 1 -- 172.21.15.7:0/477272371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd40b0bb0 con 0x55dda92fbbc0 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8dda8a2700 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d2b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009310 tx=0x7f8dcc028070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465be58add07884d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8db4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8db4ff9700 1 -- 172.21.15.7:0/477272371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc02b040 con 0x55dda92fbbc0 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8db4ff9700 1 -- 172.21.15.7:0/477272371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8dcc01d4f0 con 0x55dda92fbbc0 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8de5619b80 1 -- 172.21.15.7:0/477272371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dcc00bae0 con 0x55dda92fbbc0 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8db4ff9700 1 -- 172.21.15.7:0/477272371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dcc025810 con 0x55dda92fbbc0 2023-10-07T07:59:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8de5619b80 10 client.5523 fetch_fsmap learned FSMap version 25 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8de5619b80 1 -- 172.21.15.7:0/477272371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55dda903bd60 con 0x55dda92fbbc0 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8db4ff9700 1 -- 172.21.15.7:0/477272371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc00bae0 con 0x55dda92fbbc0 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8db4ff9700 1 -- 172.21.15.7:0/477272371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8dcc00b050 con 0x55dda92fbbc0 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8de5619b80 10 client.5523 fetch_fsmap finished waiting for FSMap version 25 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8de5619b80 10 client.5523 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8de5619b80 20 client.5523 populate_metadata read hostname 'smithi007' 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8de5619b80 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] conn(0x55dda92a1d10 0x55dda92f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8de5619b80 4 client.5523 mds_command: new command op to 5382 tid=0[{"prefix": "status"}] 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8de5619b80 1 -- 172.21.15.7:0/477272371 --> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dda919e8d0 con 0x55dda92a1d10 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.470+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] conn(0x55dda92a1d10 0x55dda92f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8db4ff9700 10 client.5523 ms_handle_connect on v2:172.21.15.7:6834/763397529 2023-10-07T07:59:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8dda0a1700 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] conn(0x55dda92a1d10 0x55dda92f9d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b3966219ddbfdf1 server_cookie=e8a37a89e6f9caa8 in_seq=0 out_seq=0 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8db4ff9700 1 -- 172.21.15.7:0/477272371 <== mds.0 v2:172.21.15.7:6834/763397529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55dda919e8d0 con 0x55dda92a1d10 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8db4ff9700 10 client.5523 handle_command_reply: tid=0 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 client.5523 shutdown 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 -- 172.21.15.7:0/477272371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 msgr2=0x55dda92d2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d2b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009310 tx=0x7f8dcc028070 comp rx=0 tx=0).stop 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 -- 172.21.15.7:0/477272371 shutdown_connections 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.7:6834/763397529,v1:172.21.15.7:6835/763397529] conn(0x55dda92a1d10 0x55dda92f9d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dda92fbbc0 0x55dda92d2b60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dda92d8530 0x55dda9347a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8db4ff9700 0 client.5523 ms_handle_reset on v2:172.21.15.7:6834/763397529 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 --2- 172.21.15.7:0/477272371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dda92f2a40 0x55dda93579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 -- 172.21.15.7:0/477272371 >> 172.21.15.7:0/477272371 conn(0x55dda92f5e70 msgr2=0x55dda92d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:56.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 -- 172.21.15.7:0/477272371 shutdown_connections 2023-10-07T07:59:56.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.471+0000 7f8de5619b80 1 -- 172.21.15.7:0/477272371 wait complete. 2023-10-07T07:59:56.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8de5619b80 20 client.5523 trim_cache size 0 max 16384 2023-10-07T07:59:56.479 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:59:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "c71d63e6-6de9-4139-83e7-b0bb7f6375d7", 2023-10-07T07:59:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2023-10-07T07:59:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5382, 2023-10-07T07:59:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2023-10-07T07:59:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2023-10-07T07:59:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2023-10-07T07:59:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1070644209999996, 2023-10-07T07:59:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 25, 2023-10-07T07:59:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 45, 2023-10-07T07:59:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 45, 2023-10-07T07:59:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.082198468 2023-10-07T07:59:56.480 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:59:56.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 -- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8db8066f30 msgr2=0x7f8db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8db8066f30 0x7f8db80693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40c8720 tx=0x7f8dd0008040 comp rx=0 tx=0).stop 2023-10-07T07:59:56.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 -- 172.21.15.7:0/4133913921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 msgr2=0x7f8dd41568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 0x7f8dd41568c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800ec80 tx=0x7f8dc80091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 -- 172.21.15.7:0/4133913921 shutdown_connections 2023-10-07T07:59:56.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8db8066f30 0x7f8db80693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4159690 0x7f8dd415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40cab80 0x7f8dd41568c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 --2- 172.21.15.7:0/4133913921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd41592c0 0x7f8dd415ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 -- 172.21.15.7:0/4133913921 >> 172.21.15.7:0/4133913921 conn(0x7f8dd4068bd0 msgr2=0x7f8dd4066ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:56.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 -- 172.21.15.7:0/4133913921 shutdown_connections 2023-10-07T07:59:56.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.472+0000 7f8ddc305700 1 -- 172.21.15.7:0/4133913921 wait complete. 2023-10-07T07:59:56.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T07:59:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.617+0000 7f8f4f489700 1 Processor -- start 2023-10-07T07:59:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.618+0000 7f8f4f489700 1 -- start start 2023-10-07T07:59:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.618+0000 7f8f4f489700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8f481570a0 0x7f8f480c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.618+0000 7f8f4f489700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f48157470 0x7f8f4806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.618+0000 7f8f4f489700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f4806bb60 0x7f8f4806cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.618+0000 7f8f4f489700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4806bfe0 con 0x7f8f4806bb60 2023-10-07T07:59:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.618+0000 7f8f4f489700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480c3b80 con 0x7f8f481570a0 2023-10-07T07:59:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.618+0000 7f8f4f489700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c2f70 con 0x7f8f48157470 2023-10-07T07:59:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4da26700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f4806bb60 0x7f8f4806cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4da26700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f4806bb60 0x7f8f4806cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38666/0 (socket says 172.21.15.7:38666) 2023-10-07T07:59:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4da26700 1 -- 172.21.15.7:0/186372517 learned_addr learned my addr 172.21.15.7:0/186372517 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4d225700 1 --2- 172.21.15.7:0/186372517 >> v2:172.21.15.172:3300/0 conn(0x7f8f481570a0 0x7f8f480c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4ca24700 1 --2- 172.21.15.7:0/186372517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f48157470 0x7f8f4806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4da26700 1 -- 172.21.15.7:0/186372517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f48157470 msgr2=0x7f8f4806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4da26700 1 --2- 172.21.15.7:0/186372517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f48157470 0x7f8f4806b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4da26700 1 -- 172.21.15.7:0/186372517 >> v2:172.21.15.172:3300/0 conn(0x7f8f481570a0 msgr2=0x7f8f480c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:56.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4da26700 1 --2- 172.21.15.7:0/186372517 >> v2:172.21.15.172:3300/0 conn(0x7f8f481570a0 0x7f8f480c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4da26700 1 -- 172.21.15.7:0/186372517 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f480c3250 con 0x7f8f4806bb60 2023-10-07T07:59:56.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f4da26700 1 --2- 172.21.15.7:0/186372517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4806bb60 0x7f8f4806cac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8f480caa70 tx=0x7f8f40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7f353d9305346b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.619+0000 7f8f3ffff700 1 -- 172.21.15.7:0/186372517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000e070 con 0x7f8f4806bb60 2023-10-07T07:59:56.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f3ffff700 1 -- 172.21.15.7:0/186372517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4000ba50 con 0x7f8f4806bb60 2023-10-07T07:59:56.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f3ffff700 1 -- 172.21.15.7:0/186372517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400092e0 con 0x7f8f4806bb60 2023-10-07T07:59:56.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f4f489700 1 -- 172.21.15.7:0/186372517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4806bb60 msgr2=0x7f8f4806cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f4f489700 1 --2- 172.21.15.7:0/186372517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4806bb60 0x7f8f4806cac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8f480caa70 tx=0x7f8f40009840 comp rx=0 tx=0).stop 2023-10-07T07:59:56.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f4f489700 1 -- 172.21.15.7:0/186372517 shutdown_connections 2023-10-07T07:59:56.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f4f489700 1 --2- 172.21.15.7:0/186372517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4806bb60 0x7f8f4806cac0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f4f489700 1 --2- 172.21.15.7:0/186372517 >> v2:172.21.15.172:3300/0 conn(0x7f8f481570a0 0x7f8f480c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f4f489700 1 --2- 172.21.15.7:0/186372517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f48157470 0x7f8f4806b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f4f489700 1 -- 172.21.15.7:0/186372517 >> 172.21.15.7:0/186372517 conn(0x7f8f48067ac0 msgr2=0x7f8f480ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:56.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f4f489700 1 -- 172.21.15.7:0/186372517 shutdown_connections 2023-10-07T07:59:56.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.620+0000 7f8f4f489700 1 -- 172.21.15.7:0/186372517 wait complete. 2023-10-07T07:59:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.621+0000 7f8f4f489700 1 Processor -- start 2023-10-07T07:59:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.621+0000 7f8f4f489700 1 -- start start 2023-10-07T07:59:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.622+0000 7f8f4f489700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f4806bb60 0x7f8f48152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.622+0000 7f8f4f489700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f481570a0 0x7f8f480b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.622+0000 7f8f4f489700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157470 0x7f8f4813ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.622+0000 7f8f4f489700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480b2330 con 0x7f8f48157470 2023-10-07T07:59:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.622+0000 7f8f4f489700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480b27c0 con 0x7f8f481570a0 2023-10-07T07:59:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.622+0000 7f8f4f489700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480b1720 con 0x7f8f4806bb60 2023-10-07T07:59:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4da26700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157470 0x7f8f4813ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4da26700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157470 0x7f8f4813ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38676/0 (socket says 172.21.15.7:38676) 2023-10-07T07:59:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4da26700 1 -- 172.21.15.7:0/1468814769 learned_addr learned my addr 172.21.15.7:0/1468814769 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:56.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4d225700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f4806bb60 0x7f8f48152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4ca24700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f481570a0 0x7f8f480b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4da26700 1 -- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f4806bb60 msgr2=0x7f8f48152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4da26700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f4806bb60 0x7f8f48152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4da26700 1 -- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f481570a0 msgr2=0x7f8f480b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4da26700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f481570a0 0x7f8f480b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4da26700 1 -- 172.21.15.7:0/1468814769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40009ae0 con 0x7f8f48157470 2023-10-07T07:59:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4ca24700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f481570a0 0x7f8f480b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4d225700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f4806bb60 0x7f8f48152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T07:59:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4da26700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157470 0x7f8f4813ecc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f480cbc20 tx=0x7f8f40018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f3d7fa700 1 -- 172.21.15.7:0/1468814769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400032f0 con 0x7f8f48157470 2023-10-07T07:59:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f3d7fa700 1 -- 172.21.15.7:0/1468814769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f40008e80 con 0x7f8f48157470 2023-10-07T07:59:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f480b1a00 con 0x7f8f48157470 2023-10-07T07:59:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f3d7fa700 1 -- 172.21.15.7:0/1468814769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400256b0 con 0x7f8f48157470 2023-10-07T07:59:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.623+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f480b1fc0 con 0x7f8f48157470 2023-10-07T07:59:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.624+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f480716c0 con 0x7f8f48157470 2023-10-07T07:59:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.625+0000 7f8f3d7fa700 1 -- 172.21.15.7:0/1468814769 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f40003040 con 0x7f8f48157470 2023-10-07T07:59:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.625+0000 7f8f3d7fa700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8f2c066f30 0x7f8f2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.625+0000 7f8f3d7fa700 1 -- 172.21.15.7:0/1468814769 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f40086270 con 0x7f8f48157470 2023-10-07T07:59:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.626+0000 7f8f4d225700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8f2c066f30 0x7f8f2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.626+0000 7f8f4d225700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8f2c066f30 0x7f8f2c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006fd0 tx=0x7f8f44008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.630+0000 7f8f3d7fa700 1 -- 172.21.15.7:0/1468814769 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f400534e0 con 0x7f8f48157470 2023-10-07T07:59:56.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.879+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8f4813a9c0 con 0x7f8f48157470 2023-10-07T07:59:56.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.880+0000 7f8f3d7fa700 1 -- 172.21.15.7:0/1468814769 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8f40009940 con 0x7f8f48157470 2023-10-07T07:59:56.881 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T07:59:56.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8f2c066f30 msgr2=0x7f8f2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8f2c066f30 0x7f8f2c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006fd0 tx=0x7f8f44008040 comp rx=0 tx=0).stop 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157470 msgr2=0x7f8f4813ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157470 0x7f8f4813ecc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f480cbc20 tx=0x7f8f40018000 comp rx=0 tx=0).stop 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 shutdown_connections 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f8f2c066f30 0x7f8f2c0693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157470 0x7f8f4813ecc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8f4806bb60 0x7f8f48152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 --2- 172.21.15.7:0/1468814769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f481570a0 0x7f8f480b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 >> 172.21.15.7:0/1468814769 conn(0x7f8f48067ac0 msgr2=0x7f8f48139af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 shutdown_connections 2023-10-07T07:59:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:56.882+0000 7f8f4f489700 1 -- 172.21.15.7:0/1468814769 wait complete. 2023-10-07T07:59:56.897 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T07:59:56.897 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T07:59:56.897 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-07T07:59:56.898 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-07T07:59:56.898 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-07T07:59:56.898 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T07:59:56.898 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T07:59:56.898 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T07:59:56.898 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T07:59:56.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:56.898 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-07T07:59:56.919 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:59:56.920 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:59:56.920 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:59:56.920 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:56.920 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-07T07:59:56.920 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:56.920 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:59:56.920 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:56.920 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2023-10-07T07:59:56.920 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42338sec preferred_lft 42338sec 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:56.921 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:59:56.922 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:56.922 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2023-10-07T07:59:56.922 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:59:56.922 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:3a:d8:7f:89:ca brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:56.922 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T07:59:56.922 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:56.922 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f00c:5eff:fefc:98cd/64 scope link 2023-10-07T07:59:56.923 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:56.923 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:59:56.923 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 62:07:64:37:78:fc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T07:59:56.923 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6007:64ff:fe37:78fc/64 scope link 2023-10-07T07:59:56.923 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:56.924 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:59:56.924 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:3a:d8:7f:89:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T07:59:56.924 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::43a:d8ff:fe7f:89ca/64 scope link 2023-10-07T07:59:56.924 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:56.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:56.925 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T07:59:56.991 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-07T07:59:56.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:56.993 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-07T07:59:57.059 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-07T07:59:57.059 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T07:59:57.059 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-07T07:59:57.059 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-07T07:59:57.059 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T07:59:57.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:57.060 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T07:59:57.060 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T07:59:57.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T07:59:57.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T07:59:57.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T07:59:57.060 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T07:59:57.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:57.182 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-07T07:59:57.204 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-07T07:59:57.204 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T07:59:57.204 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T07:59:57.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:57.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2023-10-07T07:59:57.281 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-07T07:59:57.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:59:57.281 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-07T07:59:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-07T07:59:57.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:59:57.345 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-07T07:59:57.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T07:59:57.485 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:59:57.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:59:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:59:57.562 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:59:57.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:59:57.562 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T07:59:57.626 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T07:59:57.627 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-07T07:59:57.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T07:59:57.705 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:59:57.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:59:57.771 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-10-07T07:59:57.770+0000 7fc4a29ed540 -1 init, newargv = 0x55d44916f920 newargc=15 2023-10-07T07:59:57.771 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[136798]: starting ceph client 2023-10-07T07:59:57.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[136798]: starting fuse 2023-10-07T07:59:57.777 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:59:57.779 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:59:57.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:59:57.779 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T07:59:57.843 INFO:teuthology.orchestra.run.smithi007.stdout:47 2023-10-07T07:59:57.843 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-07T07:59:57.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:59:57.843 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-07T07:59:57.843 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-07T07:59:57.843 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T07:59:57.843 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-07T07:59:57.844 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T07:59:57.845 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T07:59:57.845 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T07:59:57.845 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-07T07:59:57.845 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-07T07:59:57.845 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-07T07:59:57.845 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T07:59:57.845 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T07:59:57.845 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2023-10-07T07:59:57.845 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T07:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.136798.asok 2023-10-07T07:59:57.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:59:58.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T07:59:58.162 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.136798.asok 2023-10-07T07:59:58.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:59:58.163 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.136798.asok status 2023-10-07T07:59:58.298 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T07:59:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-07T07:59:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T07:59:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T07:59:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2023-10-07T07:59:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "136798", 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5538, 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 5538 2023-10-07T07:59:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1405293715 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1405293715 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T07:59:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.5538 192.168.0.1:0/1405293715", 2023-10-07T07:59:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/1405293715", 2023-10-07T07:59:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-07T07:59:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 25, 2023-10-07T07:59:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 45, 2023-10-07T07:59:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-07T07:59:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-07T07:59:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-07T07:59:58.301 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T07:59:58.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:59:58.309 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-07T07:59:58.309 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-07T07:59:58.330 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-07T07:59:58.330 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-07T07:59:58.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:59:58.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-07T07:59:58.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T07:59:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.585+0000 7ff78fbb6700 1 Processor -- start 2023-10-07T07:59:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.585+0000 7ff78fbb6700 1 -- start start 2023-10-07T07:59:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.586+0000 7ff78fbb6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff7881570a0 0x7ff7880c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.586+0000 7ff78fbb6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff788157470 0x7ff78806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.586+0000 7ff78fbb6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7880c3b80 con 0x7ff78806bb60 2023-10-07T07:59:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.586+0000 7ff78fbb6700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78806bfe0 con 0x7ff7881570a0 2023-10-07T07:59:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.586+0000 7ff78fbb6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880c2f70 con 0x7ff788157470 2023-10-07T07:59:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.586+0000 7ff78d952700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff7881570a0 0x7ff7880c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.586+0000 7ff78d151700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff788157470 0x7ff78806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.586+0000 7ff78e153700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff78806bb60 0x7ff78806cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:39728/0 (socket says 172.21.15.7:39728) 2023-10-07T07:59:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.586+0000 7ff78e153700 1 -- 172.21.15.7:0/3302924771 learned_addr learned my addr 172.21.15.7:0/3302924771 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T07:59:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.587+0000 7ff78d952700 1 -- 172.21.15.7:0/3302924771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff788157470 msgr2=0x7ff78806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T07:59:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.587+0000 7ff78d952700 1 --2- 172.21.15.7:0/3302924771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff788157470 0x7ff78806b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.587+0000 7ff78d952700 1 -- 172.21.15.7:0/3302924771 >> v1:172.21.15.172:6789/0 conn(0x7ff78806bb60 legacy=0x7ff78806cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.587+0000 7ff78d952700 1 -- 172.21.15.7:0/3302924771 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7880c3250 con 0x7ff7881570a0 2023-10-07T07:59:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.587+0000 7ff78d151700 1 --2- 172.21.15.7:0/3302924771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff788157470 0x7ff78806b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T07:59:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.587+0000 7ff78d952700 1 --2- 172.21.15.7:0/3302924771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7881570a0 0x7ff7880c8800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff7840098e0 tx=0x7ff7840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6977a503b60831f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.587+0000 7ff78c950700 1 -- 172.21.15.7:0/3302924771 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400b640 con 0x7ff7881570a0 2023-10-07T07:59:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.587+0000 7ff78c950700 1 -- 172.21.15.7:0/3302924771 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78400b7a0 con 0x7ff7881570a0 2023-10-07T07:59:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.587+0000 7ff78c950700 1 -- 172.21.15.7:0/3302924771 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400bb40 con 0x7ff7881570a0 2023-10-07T07:59:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.588+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3302924771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7881570a0 msgr2=0x7ff7880c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.588+0000 7ff78fbb6700 1 --2- 172.21.15.7:0/3302924771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7881570a0 0x7ff7880c8800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff7840098e0 tx=0x7ff7840091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.588+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3302924771 shutdown_connections 2023-10-07T07:59:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.588+0000 7ff78fbb6700 1 --2- 172.21.15.7:0/3302924771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7881570a0 0x7ff7880c8800 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.588+0000 7ff78fbb6700 1 --2- 172.21.15.7:0/3302924771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff788157470 0x7ff78806b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.588+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3302924771 >> 172.21.15.7:0/3302924771 conn(0x7ff788067ac0 msgr2=0x7ff7880ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.588+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3302924771 shutdown_connections 2023-10-07T07:59:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.588+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3302924771 wait complete. 2023-10-07T07:59:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.589+0000 7ff78fbb6700 1 Processor -- start 2023-10-07T07:59:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.589+0000 7ff78fbb6700 1 -- start start 2023-10-07T07:59:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78fbb6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78806bb60 0x7ff788136c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78fbb6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7881570a0 0x7ff788137db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78fbb6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff788157470 0x7ff7880c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78fbb6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7881382f0 con 0x7ff7881570a0 2023-10-07T07:59:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78fbb6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880c5590 con 0x7ff788157470 2023-10-07T07:59:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78fbb6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880c5e30 con 0x7ff78806bb60 2023-10-07T07:59:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78d151700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7881570a0 0x7ff788137db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78d151700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7881570a0 0x7ff788137db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38692/0 (socket says 172.21.15.7:38692) 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78d151700 1 -- 172.21.15.7:0/3243978871 learned_addr learned my addr 172.21.15.7:0/3243978871 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78d952700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78806bb60 0x7ff788136c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78e153700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff788157470 0x7ff7880c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78d151700 1 -- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78806bb60 msgr2=0x7ff788136c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78d151700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78806bb60 0x7ff788136c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78d151700 1 -- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff788157470 msgr2=0x7ff7880c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78d151700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff788157470 0x7ff7880c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.590+0000 7ff78d151700 1 -- 172.21.15.7:0/3243978871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff778009420 con 0x7ff7881570a0 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.591+0000 7ff78d151700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7881570a0 0x7ff788137db0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff77800ecb0 tx=0x7ff7780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.591+0000 7ff77dffb700 1 -- 172.21.15.7:0/3243978871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77800c850 con 0x7ff7881570a0 2023-10-07T07:59:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.591+0000 7ff77dffb700 1 -- 172.21.15.7:0/3243978871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77800c9b0 con 0x7ff7881570a0 2023-10-07T07:59:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.591+0000 7ff77dffb700 1 -- 172.21.15.7:0/3243978871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77800cd50 con 0x7ff7881570a0 2023-10-07T07:59:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.591+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff784009420 con 0x7ff7881570a0 2023-10-07T07:59:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.591+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7880c57e0 con 0x7ff7881570a0 2023-10-07T07:59:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.592+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7880716c0 con 0x7ff7881570a0 2023-10-07T07:59:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.593+0000 7ff77dffb700 1 -- 172.21.15.7:0/3243978871 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff778013680 con 0x7ff7881570a0 2023-10-07T07:59:58.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.593+0000 7ff77dffb700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff76c066f30 0x7ff76c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T07:59:58.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.593+0000 7ff77dffb700 1 -- 172.21.15.7:0/3243978871 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff7780846a0 con 0x7ff7881570a0 2023-10-07T07:59:58.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.593+0000 7ff78d952700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff76c066f30 0x7ff76c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T07:59:58.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.595+0000 7ff78d952700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff76c066f30 0x7ff76c0693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff7840093f0 tx=0x7ff784017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T07:59:58.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.598+0000 7ff77dffb700 1 -- 172.21.15.7:0/3243978871 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff778051910 con 0x7ff7881570a0 2023-10-07T07:59:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.841+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff7880c5f70 con 0x7ff7881570a0 2023-10-07T07:59:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.842+0000 7ff77dffb700 1 -- 172.21.15.7:0/3243978871 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff778054f30 con 0x7ff7881570a0 2023-10-07T07:59:58.843 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T07:59:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff76c066f30 msgr2=0x7ff76c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff76c066f30 0x7ff76c0693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff7840093f0 tx=0x7ff784017000 comp rx=0 tx=0).stop 2023-10-07T07:59:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7881570a0 msgr2=0x7ff788137db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T07:59:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7881570a0 0x7ff788137db0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff77800ecb0 tx=0x7ff7780091a0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 shutdown_connections 2023-10-07T07:59:58.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff76c066f30 0x7ff76c0693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff788157470 0x7ff7880c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78806bb60 0x7ff788136c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 --2- 172.21.15.7:0/3243978871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7881570a0 0x7ff788137db0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T07:59:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 >> 172.21.15.7:0/3243978871 conn(0x7ff788067ac0 msgr2=0x7ff78806cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T07:59:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 shutdown_connections 2023-10-07T07:59:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T07:59:58.844+0000 7ff78fbb6700 1 -- 172.21.15.7:0/3243978871 wait complete. 2023-10-07T07:59:58.858 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T07:59:58.858 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T07:59:58.859 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-10-07T07:59:58.859 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-07T07:59:58.859 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-07T07:59:58.859 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T07:59:58.859 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T07:59:58.859 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T07:59:58.859 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T07:59:58.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:58.859 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-10-07T07:59:58.882 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:58.883 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42324sec preferred_lft 42324sec 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 36:e1:63:77:36:7f brd ff:ff:ff:ff:ff:ff 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T07:59:58.884 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2:8eff:fee3:e22b/64 scope link 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 36:e1:63:77:36:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::34e1:63ff:fe77:367f/64 scope link 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether fe:48:36:09:05:1a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::fc48:36ff:fe09:51a/64 scope link 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-10-07T07:59:58.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:58.886 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T07:59:58.954 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-10-07T07:59:58.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:58.955 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-10-07T07:59:59.019 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-10-07T07:59:59.020 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T07:59:59.020 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T07:59:59.020 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-07T07:59:59.020 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T07:59:59.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:59.020 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-10-07T07:59:59.021 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-10-07T07:59:59.021 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T07:59:59.021 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T07:59:59.021 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T07:59:59.021 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-10-07T07:59:59.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:59.134 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-10-07T07:59:59.153 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-07T07:59:59.153 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-07T07:59:59.153 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T07:59:59.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T07:59:59.154 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2023-10-07T07:59:59.222 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-07T07:59:59.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:59:59.223 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-07T07:59:59.281 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-07T07:59:59.282 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T07:59:59.282 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-07T07:59:59.341 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2023-10-07T07:59:59.409 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:59:59.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:59:59.478 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:59:59.479 INFO:teuthology.orchestra.run.smithi172.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:59:59.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:59:59.479 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2023-10-07T07:59:59.537 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T07:59:59.537 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-07T07:59:59.580 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2023-10-07T07:59:59.606 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T07:59:59.606 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T07:59:59.661 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:2023-10-07T07:59:59.660+0000 7fede81a1540 -1 init, newargv = 0x55b33755ab20 newargc=15 2023-10-07T07:59:59.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:ceph-fuse[129102]: starting ceph client 2023-10-07T07:59:59.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:ceph-fuse[129102]: starting fuse 2023-10-07T07:59:59.676 INFO:teuthology.orchestra.run.smithi172.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T07:59:59.677 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T07:59:59.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T07:59:59.677 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2023-10-07T07:59:59.735 INFO:teuthology.orchestra.run.smithi172.stdout:46 2023-10-07T07:59:59.736 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-07T07:59:59.736 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T07:59:59.736 DEBUG:teuthology.orchestra.run.smithi172:> import glob 2023-10-07T07:59:59.736 DEBUG:teuthology.orchestra.run.smithi172:> import re 2023-10-07T07:59:59.736 DEBUG:teuthology.orchestra.run.smithi172:> import os 2023-10-07T07:59:59.736 DEBUG:teuthology.orchestra.run.smithi172:> import subprocess 2023-10-07T07:59:59.736 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:59:59.736 DEBUG:teuthology.orchestra.run.smithi172:> def _find_admin_socket(client_name): 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> files = glob.glob(asok_path) 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> # Given a non-glob path, it better be there 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> if "*" not in asok_path: 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> assert(len(files) == 1) 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> return files[0] 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> for f in files: 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T07:59:59.737 DEBUG:teuthology.orchestra.run.smithi172:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T07:59:59.738 DEBUG:teuthology.orchestra.run.smithi172:> contents = proc_f.read() 2023-10-07T07:59:59.738 DEBUG:teuthology.orchestra.run.smithi172:> if mountpoint in contents: 2023-10-07T07:59:59.738 DEBUG:teuthology.orchestra.run.smithi172:> return f 2023-10-07T07:59:59.738 DEBUG:teuthology.orchestra.run.smithi172:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T07:59:59.738 DEBUG:teuthology.orchestra.run.smithi172:> 2023-10-07T07:59:59.738 DEBUG:teuthology.orchestra.run.smithi172:> print(_find_admin_socket("client.1")) 2023-10-07T07:59:59.738 DEBUG:teuthology.orchestra.run.smithi172:> ' 2023-10-07T07:59:59.859 INFO:teuthology.orchestra.run.smithi172.stdout:/var/run/ceph/ceph-client.1.129102.asok 2023-10-07T08:00:00.041 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.129102.asok 2023-10-07T08:00:00.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T08:00:00.042 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.129102.asok status 2023-10-07T08:00:00.197 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2023-10-07T08:00:00.197 INFO:teuthology.orchestra.run.smithi172.stdout: "metadata": { 2023-10-07T08:00:00.197 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T08:00:00.197 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T08:00:00.197 INFO:teuthology.orchestra.run.smithi172.stdout: "entity_id": "1", 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": "smithi172", 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "pid": "129102", 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "root": "/" 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_count": 0, 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_pinned_count": 0, 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 5021, 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "inst": { 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "name": { 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "client", 2023-10-07T08:00:00.198 INFO:teuthology.orchestra.run.smithi172.stdout: "num": 5021 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 62767507 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: } 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 62767507 2023-10-07T08:00:00.199 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2023-10-07T08:00:00.200 INFO:teuthology.orchestra.run.smithi172.stdout: "inst_str": "client.5021 192.168.0.1:0/62767507", 2023-10-07T08:00:00.200 INFO:teuthology.orchestra.run.smithi172.stdout: "addr_str": "192.168.0.1:0/62767507", 2023-10-07T08:00:00.200 INFO:teuthology.orchestra.run.smithi172.stdout: "inode_count": 1, 2023-10-07T08:00:00.200 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_epoch": 25, 2023-10-07T08:00:00.200 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch": 45, 2023-10-07T08:00:00.200 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch_barrier": 0, 2023-10-07T08:00:00.200 INFO:teuthology.orchestra.run.smithi172.stdout: "blocklisted": false, 2023-10-07T08:00:00.200 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs" 2023-10-07T08:00:00.200 INFO:teuthology.orchestra.run.smithi172.stdout:} 2023-10-07T08:00:00.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T08:00:00.207 DEBUG:teuthology.orchestra.run.smithi172:> stat --file-system '--printf=%T 2023-10-07T08:00:00.208 DEBUG:teuthology.orchestra.run.smithi172:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-07T08:00:00.226 INFO:teuthology.orchestra.run.smithi172.stdout:fuseblk 2023-10-07T08:00:00.227 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-07T08:00:00.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T08:00:00.227 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-07T08:00:00.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set client client_inject_fixed_oldest_tid true 2023-10-07T08:00:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.443+0000 7fe6157a1700 1 Processor -- start 2023-10-07T08:00:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.443+0000 7fe6157a1700 1 -- start start 2023-10-07T08:00:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe6157a1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6101407b0 0x7fe6101581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe6157a1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe610140b80 0x7fe61006d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe6157a1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6100c7610 con 0x7fe6100c64b0 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe6157a1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61006f620 con 0x7fe610140b80 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe6157a1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe610158700 con 0x7fe6101407b0 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe60e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe610140b80 0x7fe61006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe60e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe610140b80 0x7fe61006d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36906/0 (socket says 172.21.15.7:36906) 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe60e7fc700 1 -- 172.21.15.7:0/2524074171 learned_addr learned my addr 172.21.15.7:0/2524074171 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe60effd700 1 --2- 172.21.15.7:0/2524074171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6101407b0 0x7fe6101581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe60e7fc700 1 -- 172.21.15.7:0/2524074171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6101407b0 msgr2=0x7fe6101581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe60e7fc700 1 --2- 172.21.15.7:0/2524074171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6101407b0 0x7fe6101581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe60e7fc700 1 -- 172.21.15.7:0/2524074171 >> v1:172.21.15.172:6789/0 conn(0x7fe6100c64b0 legacy=0x7fe610068d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe60e7fc700 1 -- 172.21.15.7:0/2524074171 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6100c6a00 con 0x7fe610140b80 2023-10-07T08:00:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.444+0000 7fe60e7fc700 1 --2- 172.21.15.7:0/2524074171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe610140b80 0x7fe61006d380 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe6040099b0 tx=0x7fe604009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdb1a9bb3f49bf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe60dffb700 1 -- 172.21.15.7:0/2524074171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60400b860 con 0x7fe610140b80 2023-10-07T08:00:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe60dffb700 1 -- 172.21.15.7:0/2524074171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60400b9c0 con 0x7fe610140b80 2023-10-07T08:00:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe60dffb700 1 -- 172.21.15.7:0/2524074171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60400bd60 con 0x7fe610140b80 2023-10-07T08:00:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe6157a1700 1 -- 172.21.15.7:0/2524074171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe610140b80 msgr2=0x7fe61006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe6157a1700 1 --2- 172.21.15.7:0/2524074171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe610140b80 0x7fe61006d380 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe6040099b0 tx=0x7fe604009250 comp rx=0 tx=0).stop 2023-10-07T08:00:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe6157a1700 1 -- 172.21.15.7:0/2524074171 shutdown_connections 2023-10-07T08:00:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe6157a1700 1 --2- 172.21.15.7:0/2524074171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6101407b0 0x7fe6101581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe6157a1700 1 --2- 172.21.15.7:0/2524074171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe610140b80 0x7fe61006d380 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe6157a1700 1 -- 172.21.15.7:0/2524074171 >> 172.21.15.7:0/2524074171 conn(0x7fe6100cbe70 msgr2=0x7fe6100cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe6157a1700 1 -- 172.21.15.7:0/2524074171 shutdown_connections 2023-10-07T08:00:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.445+0000 7fe6157a1700 1 -- 172.21.15.7:0/2524074171 wait complete. 2023-10-07T08:00:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.447+0000 7fe6157a1700 1 Processor -- start 2023-10-07T08:00:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.447+0000 7fe6157a1700 1 -- start start 2023-10-07T08:00:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.448+0000 7fe6157a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6100c64b0 0x7fe610138e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.448+0000 7fe6157a1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6101407b0 0x7fe61016b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.448+0000 7fe6157a1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe610140b80 0x7fe6101565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.448+0000 7fe6157a1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6100b0e10 con 0x7fe6100c64b0 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.448+0000 7fe6157a1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6100b0200 con 0x7fe6101407b0 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.448+0000 7fe6157a1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6100b04e0 con 0x7fe610140b80 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.448+0000 7fe60effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6100c64b0 0x7fe610138e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.448+0000 7fe60effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6100c64b0 0x7fe610138e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36912/0 (socket says 172.21.15.7:36912) 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.448+0000 7fe60effd700 1 -- 172.21.15.7:0/1908117736 learned_addr learned my addr 172.21.15.7:0/1908117736 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe60e7fc700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6101407b0 0x7fe61016b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe60f7fe700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe610140b80 0x7fe6101565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe60effd700 1 -- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe610140b80 msgr2=0x7fe6101565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe60effd700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe610140b80 0x7fe6101565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe60effd700 1 -- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6101407b0 msgr2=0x7fe61016b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe60effd700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6101407b0 0x7fe61016b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe60effd700 1 -- 172.21.15.7:0/1908117736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040094f0 con 0x7fe6100c64b0 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe60effd700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6100c64b0 0x7fe610138e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe60000d890 tx=0x7fe600008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe5f6ffd700 1 -- 172.21.15.7:0/1908117736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600018070 con 0x7fe6100c64b0 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe5f6ffd700 1 -- 172.21.15.7:0/1908117736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe600005a00 con 0x7fe6100c64b0 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6100b07c0 con 0x7fe6100c64b0 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe5f6ffd700 1 -- 172.21.15.7:0/1908117736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600005dc0 con 0x7fe6100c64b0 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.449+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61016ae10 con 0x7fe6100c64b0 2023-10-07T08:00:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.450+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61016b000 con 0x7fe6100c64b0 2023-10-07T08:00:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.451+0000 7fe5f6ffd700 1 -- 172.21.15.7:0/1908117736 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe60000b810 con 0x7fe6100c64b0 2023-10-07T08:00:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.451+0000 7fe5f6ffd700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5fc066f30 0x7fe5fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.451+0000 7fe5f6ffd700 1 -- 172.21.15.7:0/1908117736 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe6000846f0 con 0x7fe6100c64b0 2023-10-07T08:00:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.451+0000 7fe60e7fc700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5fc066f30 0x7fe5fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.452+0000 7fe60e7fc700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5fc066f30 0x7fe5fc0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe6040099b0 tx=0x7fe604017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.456+0000 7fe5f6ffd700 1 -- 172.21.15.7:0/1908117736 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe61016b000 con 0x7fe6100c64b0 2023-10-07T08:00:00.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.660+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=client_inject_fixed_oldest_tid}] v 0) v1 -- 0x7fe61016b000 con 0x7fe6100c64b0 2023-10-07T08:00:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.664+0000 7fe5f6ffd700 1 -- 172.21.15.7:0/1908117736 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fe600051960 con 0x7fe6100c64b0 2023-10-07T08:00:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.664+0000 7fe5f6ffd700 1 -- 172.21.15.7:0/1908117736 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_inject_fixed_oldest_tid}]=0 v17)=0 v17) v1 ==== 134+0+0 (secure 0 0 0) 0x7fe61016b000 con 0x7fe6100c64b0 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5fc066f30 msgr2=0x7fe5fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5fc066f30 0x7fe5fc0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe6040099b0 tx=0x7fe604017000 comp rx=0 tx=0).stop 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6100c64b0 msgr2=0x7fe610138e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6100c64b0 0x7fe610138e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe60000d890 tx=0x7fe600008040 comp rx=0 tx=0).stop 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 shutdown_connections 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe5fc066f30 0x7fe5fc0693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe610140b80 0x7fe6101565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6100c64b0 0x7fe610138e20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 --2- 172.21.15.7:0/1908117736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6101407b0 0x7fe61016b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.666+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 >> 172.21.15.7:0/1908117736 conn(0x7fe6100cbe70 msgr2=0x7fe610068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.667+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 shutdown_connections 2023-10-07T08:00:00.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.667+0000 7fe6157a1700 1 -- 172.21.15.7:0/1908117736 wait complete. 2023-10-07T08:00:00.681 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-07T08:00:00.681 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T08:00:00.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-07T08:00:00.719 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[136798]: fuse finished with error 0 and tester_r 0 2023-10-07T08:00:00.720 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T08:00:00.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:00.720 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-07T08:00:00.783 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T08:00:00.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.960+0000 7fee6b345700 1 Processor -- start 2023-10-07T08:00:00.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.961+0000 7fee6b345700 1 -- start start 2023-10-07T08:00:00.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.961+0000 7fee6b345700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fee64157470 0x7fee6406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.961+0000 7fee6b345700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee6406bb60 0x7fee640c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.961+0000 7fee6b345700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee640c8f10 con 0x7fee641570a0 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.961+0000 7fee6b345700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6406bfe0 con 0x7fee6406bb60 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.961+0000 7fee6b345700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee640c3b80 con 0x7fee64157470 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee698e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee6406bb60 0x7fee640c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee698e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee6406bb60 0x7fee640c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36918/0 (socket says 172.21.15.7:36918) 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee698e2700 1 -- 172.21.15.7:0/873455604 learned_addr learned my addr 172.21.15.7:0/873455604 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee688e0700 1 --2- 172.21.15.7:0/873455604 >> v2:172.21.15.172:3300/0 conn(0x7fee64157470 0x7fee6406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee698e2700 1 -- 172.21.15.7:0/873455604 >> v2:172.21.15.172:3300/0 conn(0x7fee64157470 msgr2=0x7fee6406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee698e2700 1 --2- 172.21.15.7:0/873455604 >> v2:172.21.15.172:3300/0 conn(0x7fee64157470 0x7fee6406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee698e2700 1 -- 172.21.15.7:0/873455604 >> v1:172.21.15.172:6789/0 conn(0x7fee641570a0 legacy=0x7fee640c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee698e2700 1 -- 172.21.15.7:0/873455604 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640c2f70 con 0x7fee6406bb60 2023-10-07T08:00:00.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee698e2700 1 --2- 172.21.15.7:0/873455604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee6406bb60 0x7fee640c4010 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fee6406df20 tx=0x7fee5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e023245f45c8d881 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:00.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee57fff700 1 -- 172.21.15.7:0/873455604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00e070 con 0x7fee6406bb60 2023-10-07T08:00:00.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee57fff700 1 -- 172.21.15.7:0/873455604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fee5c00ba20 con 0x7fee6406bb60 2023-10-07T08:00:00.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.962+0000 7fee57fff700 1 -- 172.21.15.7:0/873455604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0092e0 con 0x7fee6406bb60 2023-10-07T08:00:00.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.963+0000 7fee6b345700 1 -- 172.21.15.7:0/873455604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee6406bb60 msgr2=0x7fee640c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.963+0000 7fee6b345700 1 --2- 172.21.15.7:0/873455604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee6406bb60 0x7fee640c4010 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fee6406df20 tx=0x7fee5c009840 comp rx=0 tx=0).stop 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.963+0000 7fee6b345700 1 -- 172.21.15.7:0/873455604 shutdown_connections 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.963+0000 7fee6b345700 1 --2- 172.21.15.7:0/873455604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee6406bb60 0x7fee640c4010 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.963+0000 7fee6b345700 1 --2- 172.21.15.7:0/873455604 >> v2:172.21.15.172:3300/0 conn(0x7fee64157470 0x7fee6406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.963+0000 7fee6b345700 1 -- 172.21.15.7:0/873455604 >> 172.21.15.7:0/873455604 conn(0x7fee64067ac0 msgr2=0x7fee640ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.963+0000 7fee6b345700 1 -- 172.21.15.7:0/873455604 shutdown_connections 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.963+0000 7fee6b345700 1 -- 172.21.15.7:0/873455604 wait complete. 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.964+0000 7fee6b345700 1 Processor -- start 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.964+0000 7fee6b345700 1 -- start start 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.965+0000 7fee6b345700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee6406bb60 0x7fee640c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.965+0000 7fee6b345700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee641570a0 0x7fee640ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.965+0000 7fee6b345700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee64157470 0x7fee640cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.965+0000 7fee6b345700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640c5a80 con 0x7fee641570a0 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.965+0000 7fee6b345700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640c5d60 con 0x7fee6406bb60 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.965+0000 7fee6b345700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640c6040 con 0x7fee64157470 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee688e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee641570a0 0x7fee640ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee688e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee641570a0 0x7fee640ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36922/0 (socket says 172.21.15.7:36922) 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee688e0700 1 -- 172.21.15.7:0/559651779 learned_addr learned my addr 172.21.15.7:0/559651779 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee690e1700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee6406bb60 0x7fee640c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee698e2700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee64157470 0x7fee640cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee688e0700 1 -- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee64157470 msgr2=0x7fee640cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee688e0700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee64157470 0x7fee640cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee688e0700 1 -- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee6406bb60 msgr2=0x7fee640c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee688e0700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee6406bb60 0x7fee640c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee688e0700 1 -- 172.21.15.7:0/559651779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c009ae0 con 0x7fee641570a0 2023-10-07T08:00:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee688e0700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee641570a0 0x7fee640ccb60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fee6000ec80 tx=0x7fee600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee557fa700 1 -- 172.21.15.7:0/559651779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000c850 con 0x7fee641570a0 2023-10-07T08:00:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee557fa700 1 -- 172.21.15.7:0/559651779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fee6000c9b0 con 0x7fee641570a0 2023-10-07T08:00:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee640c6320 con 0x7fee641570a0 2023-10-07T08:00:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee557fa700 1 -- 172.21.15.7:0/559651779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000cd50 con 0x7fee641570a0 2023-10-07T08:00:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.966+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee640c5690 con 0x7fee641570a0 2023-10-07T08:00:00.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.967+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee640c6970 con 0x7fee641570a0 2023-10-07T08:00:00.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.968+0000 7fee557fa700 1 -- 172.21.15.7:0/559651779 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee60013680 con 0x7fee641570a0 2023-10-07T08:00:00.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.968+0000 7fee557fa700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fee48066f30 0x7fee480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:00.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.968+0000 7fee557fa700 1 -- 172.21.15.7:0/559651779 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee600847e0 con 0x7fee641570a0 2023-10-07T08:00:00.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.969+0000 7fee690e1700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fee48066f30 0x7fee480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:00.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.969+0000 7fee690e1700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fee48066f30 0x7fee480693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee580039e0 tx=0x7fee58007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:00.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:00.973+0000 7fee557fa700 1 -- 172.21.15.7:0/559651779 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee60051a50 con 0x7fee641570a0 2023-10-07T08:00:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.220+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fee640716c0 con 0x7fee641570a0 2023-10-07T08:00:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.221+0000 7fee557fa700 1 -- 172.21.15.7:0/559651779 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fee60055070 con 0x7fee641570a0 2023-10-07T08:00:01.222 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T08:00:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fee48066f30 msgr2=0x7fee480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fee48066f30 0x7fee480693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee580039e0 tx=0x7fee58007ec0 comp rx=0 tx=0).stop 2023-10-07T08:00:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee641570a0 msgr2=0x7fee640ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee641570a0 0x7fee640ccb60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fee6000ec80 tx=0x7fee600091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 shutdown_connections 2023-10-07T08:00:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fee48066f30 0x7fee480693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fee64157470 0x7fee640cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee6406bb60 0x7fee640c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 --2- 172.21.15.7:0/559651779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee641570a0 0x7fee640ccb60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 >> 172.21.15.7:0/559651779 conn(0x7fee64067ac0 msgr2=0x7fee64139c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 shutdown_connections 2023-10-07T08:00:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:01.223+0000 7fee6b345700 1 -- 172.21.15.7:0/559651779 wait complete. 2023-10-07T08:00:01.237 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T08:00:01.238 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T08:00:01.238 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-10-07T08:00:01.238 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-07T08:00:01.238 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-07T08:00:01.238 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T08:00:01.238 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T08:00:01.238 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T08:00:01.238 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T08:00:01.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:01.238 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-10-07T08:00:01.259 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T08:00:01.259 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T08:00:01.259 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-10-07T08:00:01.260 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42334sec preferred_lft 42334sec 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:3a:d8:7f:89:ca brd ff:ff:ff:ff:ff:ff 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T08:00:01.261 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f00c:5eff:fefc:98cd/64 scope link 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 62:07:64:37:78:fc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6007:64ff:fe37:78fc/64 scope link 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:3a:d8:7f:89:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::43a:d8ff:fe7f:89ca/64 scope link 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-10-07T08:00:01.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:01.263 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T08:00:01.331 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-10-07T08:00:01.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:01.333 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-10-07T08:00:01.399 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-10-07T08:00:01.399 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T08:00:01.399 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-07T08:00:01.399 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-07T08:00:01.399 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T08:00:01.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:01.399 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-10-07T08:00:01.399 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-10-07T08:00:01.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T08:00:01.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T08:00:01.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T08:00:01.400 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-10-07T08:00:01.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:01.522 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-07T08:00:01.543 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-07T08:00:01.543 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T08:00:01.544 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T08:00:01.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:01.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2023-10-07T08:00:01.621 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-07T08:00:01.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T08:00:01.621 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-07T08:00:01.684 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-07T08:00:01.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T08:00:01.685 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-07T08:00:01.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T08:00:01.825 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T08:00:01.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T08:00:01.900 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T08:00:01.901 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T08:00:01.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T08:00:01.901 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T08:00:01.965 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T08:00:01.965 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-07T08:00:02.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-10-07T08:00:02.044 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T08:00:02.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T08:00:02.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-10-07T08:00:02.108+0000 7f2c95628540 -1 init, newargv = 0x565075c60f90 newargc=15 2023-10-07T08:00:02.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[137525]: starting ceph client 2023-10-07T08:00:02.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[137525]: starting fuse 2023-10-07T08:00:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T08:00:02.119 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T08:00:02.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T08:00:02.119 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-10-07T08:00:02.180 INFO:teuthology.orchestra.run.smithi007.stdout:47 2023-10-07T08:00:02.181 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-07T08:00:02.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T08:00:02.181 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-10-07T08:00:02.181 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-10-07T08:00:02.181 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T08:00:02.181 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-10-07T08:00:02.181 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-10-07T08:00:02.182 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T08:00:02.183 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T08:00:02.183 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T08:00:02.183 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-10-07T08:00:02.183 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-10-07T08:00:02.183 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-10-07T08:00:02.183 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T08:00:02.183 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:02.183 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2023-10-07T08:00:02.183 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T08:00:02.319 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.137525.asok 2023-10-07T08:00:02.500 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.137525.asok 2023-10-07T08:00:02.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T08:00:02.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.137525.asok status 2023-10-07T08:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T08:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-10-07T08:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T08:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T08:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2023-10-07T08:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-10-07T08:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-07T08:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "137525", 2023-10-07T08:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-10-07T08:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T08:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-10-07T08:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-10-07T08:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5571, 2023-10-07T08:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-10-07T08:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-10-07T08:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-10-07T08:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 5571 2023-10-07T08:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T08:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T08:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T08:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-07T08:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2630131750 2023-10-07T08:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-10-07T08:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T08:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-10-07T08:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-10-07T08:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-10-07T08:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2630131750 2023-10-07T08:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-10-07T08:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.5571 192.168.0.1:0/2630131750", 2023-10-07T08:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/2630131750", 2023-10-07T08:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-10-07T08:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 25, 2023-10-07T08:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 45, 2023-10-07T08:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-10-07T08:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-10-07T08:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-10-07T08:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T08:00:02.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T08:00:02.758 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-10-07T08:00:02.758 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-07T08:00:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-10-07T08:00:02.782 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-07T08:00:02.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T08:00:02.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-07T08:00:02.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_completed_requests 10000 2023-10-07T08:00:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.057+0000 7f74b8f91700 1 Processor -- start 2023-10-07T08:00:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.057+0000 7f74b8f91700 1 -- start start 2023-10-07T08:00:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b8f91700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74b40b39b0 0x7f74b4068100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b8f91700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74b40b14a0 0x7f74b40afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b8f91700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b4068640 con 0x7f74b40b18f0 2023-10-07T08:00:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b8f91700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b40b1130 con 0x7f74b40b39b0 2023-10-07T08:00:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b8f91700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b415a8e0 con 0x7f74b40b14a0 2023-10-07T08:00:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b2d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f74b40b18f0 0x7f74b415d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51132/0 (socket says 172.21.15.7:51132) 2023-10-07T08:00:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b2d9d700 1 -- 172.21.15.7:0/2768015877 learned_addr learned my addr 172.21.15.7:0/2768015877 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74b40b39b0 0x7f74b4068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b1d9b700 1 --2- 172.21.15.7:0/2768015877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74b40b14a0 0x7f74b40afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b259c700 1 -- 172.21.15.7:0/2768015877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74b40b14a0 msgr2=0x7f74b40afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b259c700 1 --2- 172.21.15.7:0/2768015877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74b40b14a0 0x7f74b40afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b259c700 1 -- 172.21.15.7:0/2768015877 >> v1:172.21.15.7:6789/0 conn(0x7f74b40b18f0 legacy=0x7f74b415d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b259c700 1 -- 172.21.15.7:0/2768015877 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b4159cd0 con 0x7f74b40b39b0 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.058+0000 7f74b159a700 1 -- 172.21.15.7:0/2768015877 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887794251 0 0) 0x7f74b4068640 con 0x7f74b40b18f0 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b259c700 1 --2- 172.21.15.7:0/2768015877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b39b0 0x7f74b4068100 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f749c0098e0 tx=0x7f749c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb45a77166dac24 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b159a700 1 -- 172.21.15.7:0/2768015877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c009e80 con 0x7f74b40b39b0 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b159a700 1 -- 172.21.15.7:0/2768015877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f749c00b860 con 0x7f74b40b39b0 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b159a700 1 -- 172.21.15.7:0/2768015877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00bbe0 con 0x7f74b40b39b0 2023-10-07T08:00:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b8f91700 1 -- 172.21.15.7:0/2768015877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b39b0 msgr2=0x7f74b4068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b8f91700 1 --2- 172.21.15.7:0/2768015877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b39b0 0x7f74b4068100 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f749c0098e0 tx=0x7f749c0091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b8f91700 1 -- 172.21.15.7:0/2768015877 shutdown_connections 2023-10-07T08:00:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b8f91700 1 --2- 172.21.15.7:0/2768015877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b39b0 0x7f74b4068100 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b8f91700 1 --2- 172.21.15.7:0/2768015877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74b40b14a0 0x7f74b40afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b8f91700 1 -- 172.21.15.7:0/2768015877 >> 172.21.15.7:0/2768015877 conn(0x7f74b406d350 msgr2=0x7f74b406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b8f91700 1 -- 172.21.15.7:0/2768015877 shutdown_connections 2023-10-07T08:00:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.059+0000 7f74b8f91700 1 -- 172.21.15.7:0/2768015877 wait complete. 2023-10-07T08:00:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b8f91700 1 Processor -- start 2023-10-07T08:00:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b8f91700 1 -- start start 2023-10-07T08:00:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b8f91700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74b40b14a0 0x7f74b413fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b8f91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b18f0 0x7f74b40c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b8f91700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74b40b39b0 0x7f74b4069320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b8f91700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40cb820 con 0x7f74b40b18f0 2023-10-07T08:00:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b8f91700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40cac10 con 0x7f74b40b14a0 2023-10-07T08:00:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b8f91700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40caef0 con 0x7f74b40b39b0 2023-10-07T08:00:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b18f0 0x7f74b40c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b18f0 0x7f74b40c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36938/0 (socket says 172.21.15.7:36938) 2023-10-07T08:00:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.061+0000 7f74b1d9b700 1 -- 172.21.15.7:0/652858833 learned_addr learned my addr 172.21.15.7:0/652858833 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b259c700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74b40b14a0 0x7f74b413fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b1d9b700 1 -- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74b40b39b0 msgr2=0x7f74b4069320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b1d9b700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74b40b39b0 0x7f74b4069320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b1d9b700 1 -- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74b40b14a0 msgr2=0x7f74b413fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b1d9b700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74b40b14a0 0x7f74b413fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b1d9b700 1 -- 172.21.15.7:0/652858833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c009420 con 0x7f74b40b18f0 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b259c700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74b40b14a0 0x7f74b413fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b1d9b700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b18f0 0x7f74b40c7490 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f74a400d860 tx=0x7f74a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74aa7fc700 1 -- 172.21.15.7:0/652858833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4009610 con 0x7f74b40b18f0 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74aa7fc700 1 -- 172.21.15.7:0/652858833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f74a4009770 con 0x7f74b40b18f0 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74aa7fc700 1 -- 172.21.15.7:0/652858833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4009b30 con 0x7f74b40b18f0 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b40cb1d0 con 0x7f74b40b18f0 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.062+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b40c79d0 con 0x7f74b40b18f0 2023-10-07T08:00:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.063+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b404c3c0 con 0x7f74b40b18f0 2023-10-07T08:00:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.064+0000 7f74aa7fc700 1 -- 172.21.15.7:0/652858833 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74a401e460 con 0x7f74b40b18f0 2023-10-07T08:00:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.064+0000 7f74aa7fc700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7494066f30 0x7f74940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.064+0000 7f74aa7fc700 1 -- 172.21.15.7:0/652858833 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74a4084b20 con 0x7f74b40b18f0 2023-10-07T08:00:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.065+0000 7f74b259c700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7494066f30 0x7f74940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.065+0000 7f74b259c700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7494066f30 0x7f74940693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f749c0093a0 tx=0x7f749c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.069+0000 7f74aa7fc700 1 -- 172.21.15.7:0/652858833 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74a4051e10 con 0x7f74b40b18f0 2023-10-07T08:00:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.271+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_max_completed_requests}] v 0) v1 -- 0x7f74b40cb960 con 0x7f74b40b18f0 2023-10-07T08:00:03.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.275+0000 7f74aa7fc700 1 -- 172.21.15.7:0/652858833 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_completed_requests}]=0 v18)=0 v18) v1 ==== 128+0+0 (secure 0 0 0) 0x7f74a4055430 con 0x7f74b40b18f0 2023-10-07T08:00:03.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7494066f30 msgr2=0x7f74940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:03.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7494066f30 0x7f74940693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f749c0093a0 tx=0x7f749c0091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b18f0 msgr2=0x7f74b40c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b18f0 0x7f74b40c7490 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f74a400d860 tx=0x7f74a4008040 comp rx=0 tx=0).stop 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 shutdown_connections 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7494066f30 0x7f74940693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74b40b39b0 0x7f74b4069320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74b40b14a0 0x7f74b413fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 --2- 172.21.15.7:0/652858833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b40b18f0 0x7f74b40c7490 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 >> 172.21.15.7:0/652858833 conn(0x7f74b406d350 msgr2=0x7f74b415ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 shutdown_connections 2023-10-07T08:00:03.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:03.277+0000 7f74b8f91700 1 -- 172.21.15.7:0/652858833 wait complete. 2023-10-07T08:00:03.292 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T08:00:03.292 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T08:00:03.292 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2023-10-07T08:00:03.292 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:03.292 DEBUG:teuthology.orchestra.run.smithi007:> n = 10100 2023-10-07T08:00:03.292 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2023-10-07T08:00:03.292 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2023-10-07T08:00:03.292 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2023-10-07T08:00:03.292 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file1" 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> try: 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2023-10-07T08:00:03.293 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T08:00:03.294 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2023-10-07T08:00:03.295 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2023-10-07T08:00:03.295 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T08:00:28.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T08:00:28.070 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T08:00:41.601 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T08:00:41.601 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T08:00:41.601 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2023-10-07T08:00:41.601 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:41.601 DEBUG:teuthology.orchestra.run.smithi007:> n = 5 2023-10-07T08:00:41.601 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2023-10-07T08:00:41.601 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2023-10-07T08:00:41.601 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> try: 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2023-10-07T08:00:41.602 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2023-10-07T08:00:41.603 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2023-10-07T08:00:41.604 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T08:00:42.674 INFO:tasks.ceph_test_case:waiting 1000s for health warning matching MDS_CLIENT_OLDEST_TID 2023-10-07T08:00:42.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-10-07T08:00:42.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.855+0000 7fe8dcc8c700 1 Processor -- start 2023-10-07T08:00:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.855+0000 7fe8dcc8c700 1 -- start start 2023-10-07T08:00:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.856+0000 7fe8dcc8c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 0x7fe8d80c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.856+0000 7fe8dcc8c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8d81596e0 0x7fe8d806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.856+0000 7fe8dcc8c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d8068100 con 0x7fe8d80ced30 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.856+0000 7fe8dcc8c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d8068350 con 0x7fe8d81596e0 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.856+0000 7fe8dcc8c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8068630 con 0x7fe8d8159310 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8d81596e0 0x7fe8d806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8d81596e0 0x7fe8d806b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50350/0 (socket says 172.21.15.7:50350) 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d6d9d700 1 -- 172.21.15.7:0/1510299217 learned_addr learned my addr 172.21.15.7:0/1510299217 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d659c700 1 --2- 172.21.15.7:0/1510299217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 0x7fe8d80c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d6d9d700 1 -- 172.21.15.7:0/1510299217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 msgr2=0x7fe8d80c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d6d9d700 1 --2- 172.21.15.7:0/1510299217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 0x7fe8d80c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d6d9d700 1 -- 172.21.15.7:0/1510299217 >> v1:172.21.15.172:6789/0 conn(0x7fe8d80ced30 legacy=0x7fe8d80c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d6d9d700 1 -- 172.21.15.7:0/1510299217 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d8068910 con 0x7fe8d81596e0 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d6d9d700 1 --2- 172.21.15.7:0/1510299217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 0x7fe8d806b620 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009f20 tx=0x7fe8d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04212e222c6d37f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d559a700 1 -- 172.21.15.7:0/1510299217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000e070 con 0x7fe8d81596e0 2023-10-07T08:00:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d559a700 1 -- 172.21.15.7:0/1510299217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fe8d000ba80 con 0x7fe8d81596e0 2023-10-07T08:00:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.857+0000 7fe8d559a700 1 -- 172.21.15.7:0/1510299217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00092e0 con 0x7fe8d81596e0 2023-10-07T08:00:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.858+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/1510299217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 msgr2=0x7fe8d806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.858+0000 7fe8dcc8c700 1 --2- 172.21.15.7:0/1510299217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 0x7fe8d806b620 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009f20 tx=0x7fe8d0009840 comp rx=0 tx=0).stop 2023-10-07T08:00:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.858+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/1510299217 shutdown_connections 2023-10-07T08:00:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.858+0000 7fe8dcc8c700 1 --2- 172.21.15.7:0/1510299217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 0x7fe8d806b620 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.858+0000 7fe8dcc8c700 1 --2- 172.21.15.7:0/1510299217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 0x7fe8d80c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.858+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/1510299217 >> 172.21.15.7:0/1510299217 conn(0x7fe8d8067ac0 msgr2=0x7fe8d80ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.858+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/1510299217 shutdown_connections 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.858+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/1510299217 wait complete. 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.859+0000 7fe8dcc8c700 1 Processor -- start 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.859+0000 7fe8dcc8c700 1 -- start start 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8dcc8c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 0x7fe8d8159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8dcc8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 0x7fe8d815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8dcc8c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d815a760 0x7fe8d815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8dcc8c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80b12d0 con 0x7fe8d81596e0 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8dcc8c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80c6180 con 0x7fe8d815a760 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8dcc8c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80c6460 con 0x7fe8d8159310 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8d5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 0x7fe8d815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8d5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 0x7fe8d815a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50356/0 (socket says 172.21.15.7:50356) 2023-10-07T08:00:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8d5d9b700 1 -- 172.21.15.7:0/103317497 learned_addr learned my addr 172.21.15.7:0/103317497 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8d659c700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 0x7fe8d8159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.860+0000 7fe8d6d9d700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d815a760 0x7fe8d815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8d5d9b700 1 -- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 msgr2=0x7fe8d8159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8d5d9b700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 0x7fe8d8159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8d5d9b700 1 -- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d815a760 msgr2=0x7fe8d815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8d5d9b700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d815a760 0x7fe8d815abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8d5d9b700 1 -- 172.21.15.7:0/103317497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d0009ae0 con 0x7fe8d81596e0 2023-10-07T08:00:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8d5d9b700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 0x7fe8d815a220 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c4e0 tx=0x7fe8cc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:42.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8c67fc700 1 -- 172.21.15.7:0/103317497 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc011590 con 0x7fe8d81596e0 2023-10-07T08:00:42.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8c67fc700 1 -- 172.21.15.7:0/103317497 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fe8cc011710 con 0x7fe8d81596e0 2023-10-07T08:00:42.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d80c6740 con 0x7fe8d81596e0 2023-10-07T08:00:42.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8c67fc700 1 -- 172.21.15.7:0/103317497 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc011ad0 con 0x7fe8d81596e0 2023-10-07T08:00:42.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.861+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d8151f20 con 0x7fe8d81596e0 2023-10-07T08:00:42.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.862+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d80c6d50 con 0x7fe8d81596e0 2023-10-07T08:00:42.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.862+0000 7fe8c67fc700 1 -- 172.21.15.7:0/103317497 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8cc01b3b0 con 0x7fe8d81596e0 2023-10-07T08:00:42.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.863+0000 7fe8c67fc700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8b8066f30 0x7fe8b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:42.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.863+0000 7fe8c67fc700 1 -- 172.21.15.7:0/103317497 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fe8cc083aa0 con 0x7fe8d81596e0 2023-10-07T08:00:42.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.863+0000 7fe8d659c700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8b8066f30 0x7fe8b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:42.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.864+0000 7fe8d659c700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8b8066f30 0x7fe8b80693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8006fd0 tx=0x7fe8c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:42.867+0000 7fe8c67fc700 1 -- 172.21.15.7:0/103317497 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8cc050d00 con 0x7fe8d81596e0 2023-10-07T08:00:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.170+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7fe8d80716c0 con 0x7fe8d81596e0 2023-10-07T08:00:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.170+0000 7fe8c67fc700 1 -- 172.21.15.7:0/103317497 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+46 (secure 0 0 0) 0x7fe8cc017070 con 0x7fe8d81596e0 2023-10-07T08:00:43.171 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:00:43.171 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-07T08:00:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.172+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8b8066f30 msgr2=0x7fe8b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.172+0000 7fe8dcc8c700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8b8066f30 0x7fe8b80693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8006fd0 tx=0x7fe8c8008040 comp rx=0 tx=0).stop 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.172+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 msgr2=0x7fe8d815a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.172+0000 7fe8dcc8c700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 0x7fe8d815a220 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c4e0 tx=0x7fe8cc00c8a0 comp rx=0 tx=0).stop 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.172+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 shutdown_connections 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.172+0000 7fe8dcc8c700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fe8b8066f30 0x7fe8b80693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.172+0000 7fe8dcc8c700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8d815a760 0x7fe8d815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.172+0000 7fe8dcc8c700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8d8159310 0x7fe8d8159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.173+0000 7fe8dcc8c700 1 --2- 172.21.15.7:0/103317497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d81596e0 0x7fe8d815a220 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.173+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 >> 172.21.15.7:0/103317497 conn(0x7fe8d8067ac0 msgr2=0x7fe8d80c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.173+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 shutdown_connections 2023-10-07T08:00:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:43.173+0000 7fe8dcc8c700 1 -- 172.21.15.7:0/103317497 wait complete. 2023-10-07T08:00:43.189 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-07T08:00:43.189 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=1000 retry_count=0)... 2023-10-07T08:00:48.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-10-07T08:00:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.331+0000 7f16a5e2d700 1 Processor -- start 2023-10-07T08:00:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.331+0000 7f16a5e2d700 1 -- start start 2023-10-07T08:00:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.331+0000 7f16a5e2d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16a0159310 0x7f16a00c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.331+0000 7f16a5e2d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f16a01596e0 0x7f16a006b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.331+0000 7f16a5e2d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a0068100 con 0x7f16a00ced30 2023-10-07T08:00:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.331+0000 7f16a5e2d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a0068350 con 0x7f16a0159310 2023-10-07T08:00:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.331+0000 7f16a5e2d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a0068630 con 0x7f16a01596e0 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16a0159310 0x7f16a00c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16a0159310 0x7f16a00c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50366/0 (socket says 172.21.15.7:50366) 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169f7fe700 1 -- 172.21.15.7:0/4129434687 learned_addr learned my addr 172.21.15.7:0/4129434687 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169ffff700 1 --2- 172.21.15.7:0/4129434687 >> v2:172.21.15.172:3300/0 conn(0x7f16a01596e0 0x7f16a006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169f7fe700 1 -- 172.21.15.7:0/4129434687 >> v2:172.21.15.172:3300/0 conn(0x7f16a01596e0 msgr2=0x7f16a006b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169f7fe700 1 --2- 172.21.15.7:0/4129434687 >> v2:172.21.15.172:3300/0 conn(0x7f16a01596e0 0x7f16a006b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169f7fe700 1 -- 172.21.15.7:0/4129434687 >> v1:172.21.15.172:6789/0 conn(0x7f16a00ced30 legacy=0x7f16a00c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169f7fe700 1 -- 172.21.15.7:0/4129434687 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0068910 con 0x7f16a0159310 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169ffff700 1 --2- 172.21.15.7:0/4129434687 >> v2:172.21.15.172:3300/0 conn(0x7f16a01596e0 0x7f16a006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169f7fe700 1 --2- 172.21.15.7:0/4129434687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a0159310 0x7f16a00c86e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f16a00c85e0 tx=0x7f1690009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae34d60e64d20b93 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169e7fc700 1 -- 172.21.15.7:0/4129434687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000b860 con 0x7f16a0159310 2023-10-07T08:00:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169e7fc700 1 -- 172.21.15.7:0/4129434687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f169000b9c0 con 0x7f16a0159310 2023-10-07T08:00:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.332+0000 7f169e7fc700 1 -- 172.21.15.7:0/4129434687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000bd60 con 0x7f16a0159310 2023-10-07T08:00:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.333+0000 7f16a5e2d700 1 -- 172.21.15.7:0/4129434687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a0159310 msgr2=0x7f16a00c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.333+0000 7f16a5e2d700 1 --2- 172.21.15.7:0/4129434687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a0159310 0x7f16a00c86e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f16a00c85e0 tx=0x7f1690009250 comp rx=0 tx=0).stop 2023-10-07T08:00:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.333+0000 7f16a5e2d700 1 -- 172.21.15.7:0/4129434687 shutdown_connections 2023-10-07T08:00:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.333+0000 7f16a5e2d700 1 --2- 172.21.15.7:0/4129434687 >> v2:172.21.15.172:3300/0 conn(0x7f16a01596e0 0x7f16a006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.333+0000 7f16a5e2d700 1 --2- 172.21.15.7:0/4129434687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a0159310 0x7f16a00c86e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.333+0000 7f16a5e2d700 1 -- 172.21.15.7:0/4129434687 >> 172.21.15.7:0/4129434687 conn(0x7f16a0067ac0 msgr2=0x7f16a00ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.333+0000 7f16a5e2d700 1 -- 172.21.15.7:0/4129434687 shutdown_connections 2023-10-07T08:00:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.333+0000 7f16a5e2d700 1 -- 172.21.15.7:0/4129434687 wait complete. 2023-10-07T08:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.334+0000 7f16a5e2d700 1 Processor -- start 2023-10-07T08:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f16a5e2d700 1 -- start start 2023-10-07T08:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f16a5e2d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a0159310 0x7f16a016ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f16a5e2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a01596e0 0x7f16a00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f16a5e2d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a01699b0 0x7f16a00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f16a5e2d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a016aa10 con 0x7f16a01596e0 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f16a5e2d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00c6850 con 0x7f16a01699b0 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f16a5e2d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a016dd70 con 0x7f16a0159310 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f169effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a01596e0 0x7f16a00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f169effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a01596e0 0x7f16a00c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50370/0 (socket says 172.21.15.7:50370) 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.335+0000 7f169effd700 1 -- 172.21.15.7:0/2100768049 learned_addr learned my addr 172.21.15.7:0/2100768049 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169ffff700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a01699b0 0x7f16a00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169f7fe700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a0159310 0x7f16a016ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169effd700 1 -- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a0159310 msgr2=0x7f16a016ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169effd700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a0159310 0x7f16a016ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169effd700 1 -- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a01699b0 msgr2=0x7f16a00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169effd700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a01699b0 0x7f16a00b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169effd700 1 -- 172.21.15.7:0/2100768049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16900094f0 con 0x7f16a01596e0 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169f7fe700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a0159310 0x7f16a016ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169ffff700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a01699b0 0x7f16a00b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f169effd700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a01596e0 0x7f16a00c9480 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f169400c4e0 tx=0x7f169400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f16877fe700 1 -- 172.21.15.7:0/2100768049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694011590 con 0x7f16a01596e0 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f16877fe700 1 -- 172.21.15.7:0/2100768049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f1694011710 con 0x7f16a01596e0 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f16877fe700 1 -- 172.21.15.7:0/2100768049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694011ad0 con 0x7f16a01596e0 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a00c6df0 con 0x7f16a01596e0 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.336+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a016da90 con 0x7f16a01596e0 2023-10-07T08:00:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.337+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a00c6f80 con 0x7f16a01596e0 2023-10-07T08:00:48.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.338+0000 7f16877fe700 1 -- 172.21.15.7:0/2100768049 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f169401c440 con 0x7f16a01596e0 2023-10-07T08:00:48.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.338+0000 7f16877fe700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f168c066f30 0x7f168c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.338+0000 7f16877fe700 1 -- 172.21.15.7:0/2100768049 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f1694083a90 con 0x7f16a01596e0 2023-10-07T08:00:48.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.339+0000 7f169f7fe700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f168c066f30 0x7f168c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.339+0000 7f169f7fe700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f168c066f30 0x7f168c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16900090b0 tx=0x7f1690009250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.343+0000 7f16877fe700 1 -- 172.21.15.7:0/2100768049 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1694050cf0 con 0x7f16a01596e0 2023-10-07T08:00:48.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.629+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f16a00716c0 con 0x7f16a01596e0 2023-10-07T08:00:48.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.630+0000 7f16877fe700 1 -- 172.21.15.7:0/2100768049 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+322 (secure 0 0 0) 0x7f1694017070 con 0x7f16a01596e0 2023-10-07T08:00:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:00:48.632 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_OLDEST_TID":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to advance oldest client/flush tid","count":1},"detail":[{"message":"mds.c(mds.0): Client smithi007:0 failing to advance its oldest client/flush tid. client_id: 5571"}],"muted":false}},"mutes":[]} 2023-10-07T08:00:48.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f168c066f30 msgr2=0x7f168c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f168c066f30 0x7f168c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16900090b0 tx=0x7f1690009250 comp rx=0 tx=0).stop 2023-10-07T08:00:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a01596e0 msgr2=0x7f16a00c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a01596e0 0x7f16a00c9480 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f169400c4e0 tx=0x7f169400c8a0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 shutdown_connections 2023-10-07T08:00:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f168c066f30 0x7f168c0693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a01699b0 0x7f16a00b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a0159310 0x7f16a016ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 --2- 172.21.15.7:0/2100768049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a01596e0 0x7f16a00c9480 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 >> 172.21.15.7:0/2100768049 conn(0x7f16a0067ac0 msgr2=0x7f16a006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 shutdown_connections 2023-10-07T08:00:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.632+0000 7f16a5e2d700 1 -- 172.21.15.7:0/2100768049 wait complete. 2023-10-07T08:00:48.646 DEBUG:tasks.ceph_test_case:detail_strings: ['mds.c(mds.0): Client smithi007:0 failing to advance its oldest client/flush tid. client_id: 5571'] 2023-10-07T08:00:48.646 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-07T08:00:48.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_metadata_threshold 200000 2023-10-07T08:00:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.781+0000 7f522f9bd700 1 Processor -- start 2023-10-07T08:00:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.781+0000 7f522f9bd700 1 -- start start 2023-10-07T08:00:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.782+0000 7f522f9bd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5228157300 0x7f52280afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.782+0000 7f522f9bd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52280b18d0 con 0x7f52280affc0 2023-10-07T08:00:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.782+0000 7f522f9bd700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52280b0fa0 con 0x7f5228156f30 2023-10-07T08:00:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.782+0000 7f522f9bd700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52280b0440 con 0x7f5228157300 2023-10-07T08:00:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.782+0000 7f522df5a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f52280affc0 0x7f52280b3060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39748/0 (socket says 172.21.15.7:39748) 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.782+0000 7f522df5a700 1 -- 172.21.15.7:0/587815786 learned_addr learned my addr 172.21.15.7:0/587815786 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.782+0000 7f522cf58700 1 --2- 172.21.15.7:0/587815786 >> v2:172.21.15.172:3300/0 conn(0x7f5228157300 0x7f52280afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.782+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763956520 0 0) 0x7f52280b18d0 con 0x7f52280affc0 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.782+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5214003570 con 0x7f52280affc0 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 574783682 0 0) 0x7f5214003570 con 0x7f52280affc0 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 >> v2:172.21.15.172:3300/0 conn(0x7f5228157300 msgr2=0x7f52280afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 --2- 172.21.15.7:0/587815786 >> v2:172.21.15.172:3300/0 conn(0x7f5228157300 0x7f52280afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 >> v1:172.21.15.172:6789/0 conn(0x7f5228156f30 legacy=0x7f52280b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280b1280 con 0x7f52280affc0 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f52200017f0 con 0x7f52280affc0 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 >> v1:172.21.15.7:6789/0 conn(0x7f52280affc0 legacy=0x7f52280b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5214003a60 0x7f5214003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5214004810 0x7f5214004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 0x7f521400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52280b0440 con 0x7f5214004810 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521400c160 con 0x7f52140051c0 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521400c320 con 0x7f5214003a60 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f522df5a700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 0x7f521400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f522cf58700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5214004810 0x7f5214004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f522d759700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5214003a60 0x7f5214003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f522df5a700 1 -- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5214003a60 msgr2=0x7f5214003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f522df5a700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5214003a60 0x7f5214003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f522df5a700 1 -- 172.21.15.7:0/587815786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5214004810 msgr2=0x7f5214004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f522df5a700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5214004810 0x7f5214004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f522df5a700 1 -- 172.21.15.7:0/587815786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214003570 con 0x7f52140051c0 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.783+0000 7f522cf58700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5214004810 0x7f5214004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f522df5a700 1 -- 172.21.15.7:0/587815786 reap_dead start 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f522df5a700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 0x7f521400ba60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f522000a900 tx=0x7f52280b1840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb49fa26fe9be53 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200070d0 con 0x7f52140051c0 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f5220007230 con 0x7f52140051c0 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f521bfff700 1 -- 172.21.15.7:0/587815786 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200075f0 con 0x7f52140051c0 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f522f9bd700 1 -- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 msgr2=0x7f521400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f522f9bd700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 0x7f521400ba60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f522000a900 tx=0x7f52280b1840 comp rx=0 tx=0).stop 2023-10-07T08:00:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f522f9bd700 1 -- 172.21.15.7:0/587815786 shutdown_connections 2023-10-07T08:00:48.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f522f9bd700 1 --2- 172.21.15.7:0/587815786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 0x7f521400ba60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.784+0000 7f522f9bd700 1 -- 172.21.15.7:0/587815786 >> 172.21.15.7:0/587815786 conn(0x7f522806d350 msgr2=0x7f522806d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:48.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.785+0000 7f522f9bd700 1 -- 172.21.15.7:0/587815786 shutdown_connections 2023-10-07T08:00:48.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.785+0000 7f522f9bd700 1 -- 172.21.15.7:0/587815786 wait complete. 2023-10-07T08:00:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.786+0000 7f522f9bd700 1 Processor -- start 2023-10-07T08:00:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.786+0000 7f522f9bd700 1 -- start start 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522f9bd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 0x7f5228153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522f9bd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5228069320 0x7f52281406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522f9bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5228069790 0x7f5228140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522f9bd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522815bf00 con 0x7f5228069790 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522f9bd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522815c0e0 con 0x7f52140051c0 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522f9bd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522815c3c0 con 0x7f5228069320 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522df5a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5228069790 0x7f5228140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522df5a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5228069790 0x7f5228140c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50396/0 (socket says 172.21.15.7:50396) 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522df5a700 1 -- 172.21.15.7:0/1323389767 learned_addr learned my addr 172.21.15.7:0/1323389767 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522cf58700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5228069320 0x7f52281406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522d759700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 0x7f5228153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522df5a700 1 -- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5228069320 msgr2=0x7f52281406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522df5a700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5228069320 0x7f52281406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522df5a700 1 -- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 msgr2=0x7f5228153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522df5a700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 0x7f5228153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522df5a700 1 -- 172.21.15.7:0/1323389767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522815c6a0 con 0x7f5228069790 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.787+0000 7f522df5a700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5228069790 0x7f5228140c30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f522000a3f0 tx=0x7f52280b1840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.788+0000 7f52197fa700 1 -- 172.21.15.7:0/1323389767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200070d0 con 0x7f5228069790 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.788+0000 7f52197fa700 1 -- 172.21.15.7:0/1323389767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f52200017b0 con 0x7f5228069790 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.788+0000 7f52197fa700 1 -- 172.21.15.7:0/1323389767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220001920 con 0x7f5228069790 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.788+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522815c980 con 0x7f5228069790 2023-10-07T08:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.788+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52281542b0 con 0x7f5228069790 2023-10-07T08:00:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.789+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522804c3c0 con 0x7f5228069790 2023-10-07T08:00:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.789+0000 7f52197fa700 1 -- 172.21.15.7:0/1323389767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f522001a040 con 0x7f5228069790 2023-10-07T08:00:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.790+0000 7f52197fa700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f520c066f30 0x7f520c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.790+0000 7f52197fa700 1 -- 172.21.15.7:0/1323389767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f5220086810 con 0x7f5228069790 2023-10-07T08:00:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.790+0000 7f522d759700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f520c066f30 0x7f520c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.791+0000 7f522d759700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f520c066f30 0x7f520c0693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f522815a910 tx=0x7f5224007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.795+0000 7f52197fa700 1 -- 172.21.15.7:0/1323389767 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52200539f0 con 0x7f5228069790 2023-10-07T08:00:48.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:48.996+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_session_metadata_threshold}] v 0) v1 -- 0x7f52280b1560 con 0x7f5228069790 2023-10-07T08:00:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.000+0000 7f52197fa700 1 -- 172.21.15.7:0/1323389767 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_metadata_threshold}]=0 v19)=0 v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7f52200075b0 con 0x7f5228069790 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.002+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f520c066f30 msgr2=0x7f520c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.002+0000 7f522f9bd700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f520c066f30 0x7f520c0693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f522815a910 tx=0x7f5224007d30 comp rx=0 tx=0).stop 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5228069790 msgr2=0x7f5228140c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5228069790 0x7f5228140c30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f522000a3f0 tx=0x7f52280b1840 comp rx=0 tx=0).stop 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 shutdown_connections 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f520c066f30 0x7f520c0693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5228069790 0x7f5228140c30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52140051c0 0x7f5228153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 --2- 172.21.15.7:0/1323389767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5228069320 0x7f52281406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 >> 172.21.15.7:0/1323389767 conn(0x7f522806d350 msgr2=0x7f5228152480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:49.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 shutdown_connections 2023-10-07T08:00:49.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:49.003+0000 7f522f9bd700 1 -- 172.21.15.7:0/1323389767 wait complete. 2023-10-07T08:00:49.017 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T08:00:49.017 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-10-07T08:00:49.017 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2023-10-07T08:00:49.017 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:49.017 DEBUG:teuthology.orchestra.run.smithi007:> n = 20 2023-10-07T08:00:49.017 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2023-10-07T08:00:49.017 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> try: 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2023-10-07T08:00:49.018 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T08:00:49.019 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2023-10-07T08:00:49.020 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2023-10-07T08:00:49.020 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2023-10-07T08:00:49.020 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2023-10-07T08:00:49.020 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-10-07T08:00:49.484 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-07T08:00:49.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T08:00:49.572 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T08:00:49.622 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-07T08:00:49.622 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2023-10-07T08:00:49.622 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T08:00:49.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-07T08:00:49.670 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[137525]: fuse finished with error 0 and tester_r 0 2023-10-07T08:00:49.696 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T08:00:49.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:49.697 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-07T08:00:49.878 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi172.front.sepia.ceph.com... 2023-10-07T08:00:49.878 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T08:00:49.878 DEBUG:teuthology.orchestra.run.smithi172:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-07T08:00:49.922 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi172.stderr:ceph-fuse[129102]: fuse finished with error 0 and tester_r 0 2023-10-07T08:00:49.923 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T08:00:49.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:49.924 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-07T08:00:50.098 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T08:00:50.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:50.099 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-07T08:00:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-07T08:00:50.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T08:00:50.123 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T08:00:50.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:00:50.123 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-07T08:00:50.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T08:00:50.161 INFO:teuthology.orchestra.run.smithi172.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-07T08:00:50.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T08:00:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.305+0000 7ffb63c3d700 1 Processor -- start 2023-10-07T08:00:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.305+0000 7ffb63c3d700 1 -- start start 2023-10-07T08:00:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb63c3d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 0x7ffb5c0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb63c3d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb5c1562f0 0x7ffb5c06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb63c3d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffb5c06e2c0 0x7ffb5c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb63c3d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5c06ec60 con 0x7ffb5c1562f0 2023-10-07T08:00:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb63c3d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5c06b700 con 0x7ffb5c06e2c0 2023-10-07T08:00:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb63c3d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c06aaf0 con 0x7ffb5c155f20 2023-10-07T08:00:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb611d8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb5c1562f0 0x7ffb5c06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb611d8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb5c1562f0 0x7ffb5c06a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47914/0 (socket says 172.21.15.7:47914) 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb611d8700 1 -- 172.21.15.7:0/518822906 learned_addr learned my addr 172.21.15.7:0/518822906 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb619d9700 1 --2- 172.21.15.7:0/518822906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 0x7ffb5c0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb621da700 1 --2- 172.21.15.7:0/518822906 >> v2:172.21.15.172:3300/0 conn(0x7ffb5c06e2c0 0x7ffb5c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb611d8700 1 -- 172.21.15.7:0/518822906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 msgr2=0x7ffb5c0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb611d8700 1 --2- 172.21.15.7:0/518822906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 0x7ffb5c0c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb611d8700 1 -- 172.21.15.7:0/518822906 >> v2:172.21.15.172:3300/0 conn(0x7ffb5c06e2c0 msgr2=0x7ffb5c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb611d8700 1 --2- 172.21.15.7:0/518822906 >> v2:172.21.15.172:3300/0 conn(0x7ffb5c06e2c0 0x7ffb5c06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.306+0000 7ffb611d8700 1 -- 172.21.15.7:0/518822906 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c06add0 con 0x7ffb5c1562f0 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.307+0000 7ffb611d8700 1 --2- 172.21.15.7:0/518822906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 0x7ffb5c06a3f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c06fa70 tx=0x7ffb4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b4654681fabcee server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.307+0000 7ffb609d7700 1 -- 172.21.15.7:0/518822906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c00b640 con 0x7ffb5c1562f0 2023-10-07T08:00:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.307+0000 7ffb609d7700 1 -- 172.21.15.7:0/518822906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7ffb4c00b7a0 con 0x7ffb5c1562f0 2023-10-07T08:00:50.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.307+0000 7ffb609d7700 1 -- 172.21.15.7:0/518822906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c00bb60 con 0x7ffb5c1562f0 2023-10-07T08:00:50.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.308+0000 7ffb63c3d700 1 -- 172.21.15.7:0/518822906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 msgr2=0x7ffb5c06a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.308+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/518822906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 0x7ffb5c06a3f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c06fa70 tx=0x7ffb4c009250 comp rx=0 tx=0).stop 2023-10-07T08:00:50.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.308+0000 7ffb63c3d700 1 -- 172.21.15.7:0/518822906 shutdown_connections 2023-10-07T08:00:50.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.308+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/518822906 >> v2:172.21.15.172:3300/0 conn(0x7ffb5c06e2c0 0x7ffb5c06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.308+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/518822906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 0x7ffb5c0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.308+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/518822906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 0x7ffb5c06a3f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.308+0000 7ffb63c3d700 1 -- 172.21.15.7:0/518822906 >> 172.21.15.7:0/518822906 conn(0x7ffb5c068bd0 msgr2=0x7ffb5c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.308+0000 7ffb63c3d700 1 -- 172.21.15.7:0/518822906 shutdown_connections 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.308+0000 7ffb63c3d700 1 -- 172.21.15.7:0/518822906 wait complete. 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.309+0000 7ffb63c3d700 1 Processor -- start 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.309+0000 7ffb63c3d700 1 -- start start 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb63c3d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c06e2c0 0x7ffb5c144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb63c3d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 0x7ffb5c0c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb63c3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 0x7ffb5c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb63c3d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c0c6d70 con 0x7ffb5c1562f0 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb63c3d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c14b130 con 0x7ffb5c06e2c0 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb63c3d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c14a840 con 0x7ffb5c155f20 2023-10-07T08:00:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb621da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 0x7ffb5c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb621da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 0x7ffb5c1428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47926/0 (socket says 172.21.15.7:47926) 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb621da700 1 -- 172.21.15.7:0/3590458164 learned_addr learned my addr 172.21.15.7:0/3590458164 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb619d9700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c06e2c0 0x7ffb5c144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb611d8700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 0x7ffb5c0c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb621da700 1 -- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 msgr2=0x7ffb5c0c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb621da700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 0x7ffb5c0c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb621da700 1 -- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c06e2c0 msgr2=0x7ffb5c144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb621da700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c06e2c0 0x7ffb5c144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.310+0000 7ffb621da700 1 -- 172.21.15.7:0/3590458164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c0094f0 con 0x7ffb5c1562f0 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.311+0000 7ffb621da700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 0x7ffb5c1428f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009df0 tx=0x7ffb5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.311+0000 7ffb51ffb700 1 -- 172.21.15.7:0/3590458164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400caa0 con 0x7ffb5c1562f0 2023-10-07T08:00:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.311+0000 7ffb51ffb700 1 -- 172.21.15.7:0/3590458164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7ffb5400f040 con 0x7ffb5c1562f0 2023-10-07T08:00:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.311+0000 7ffb51ffb700 1 -- 172.21.15.7:0/3590458164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400ce70 con 0x7ffb5c1562f0 2023-10-07T08:00:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.311+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c0c67f0 con 0x7ffb5c1562f0 2023-10-07T08:00:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.311+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c14aae0 con 0x7ffb5c1562f0 2023-10-07T08:00:50.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.312+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5c14b2c0 con 0x7ffb5c1562f0 2023-10-07T08:00:50.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.313+0000 7ffb51ffb700 1 -- 172.21.15.7:0/3590458164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb54014690 con 0x7ffb5c1562f0 2023-10-07T08:00:50.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.313+0000 7ffb51ffb700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffb40066f30 0x7ffb400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.313+0000 7ffb51ffb700 1 -- 172.21.15.7:0/3590458164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7ffb54085180 con 0x7ffb5c1562f0 2023-10-07T08:00:50.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.313+0000 7ffb619d9700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffb40066f30 0x7ffb400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.314+0000 7ffb619d9700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffb40066f30 0x7ffb400693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009db0 tx=0x7ffb580091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:50.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.317+0000 7ffb51ffb700 1 -- 172.21.15.7:0/3590458164 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb54055e70 con 0x7ffb5c1562f0 2023-10-07T08:00:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.568+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb5c0716c0 con 0x7ffb5c1562f0 2023-10-07T08:00:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.568+0000 7ffb51ffb700 1 -- 172.21.15.7:0/3590458164 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4589 (secure 0 0 0) 0x7ffb54055a00 con 0x7ffb5c1562f0 2023-10-07T08:00:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:00:50.569 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3622385241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3622385241},{"type":"v1","addr":"172.21.15.172:6837","nonce":3622385241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/899764672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":899764672},{"type":"v1","addr":"172.21.15.172:6836","nonce":899764672}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/318366060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":318366060},{"type":"v1","addr":"172.21.15.7:6833","nonce":318366060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:59:49.797329+0000","modified":"2023-10-07T08:00:46.532591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.7:6835/763397529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":763397529},{"type":"v1","addr":"172.21.15.7:6835","nonce":763397529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T08:00:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffb40066f30 msgr2=0x7ffb400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffb40066f30 0x7ffb400693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009db0 tx=0x7ffb580091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 msgr2=0x7ffb5c1428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 0x7ffb5c1428f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009df0 tx=0x7ffb5400b040 comp rx=0 tx=0).stop 2023-10-07T08:00:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 shutdown_connections 2023-10-07T08:00:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ffb40066f30 0x7ffb400693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5c1562f0 0x7ffb5c1428f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c06e2c0 0x7ffb5c144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 --2- 172.21.15.7:0/3590458164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c155f20 0x7ffb5c0c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 >> 172.21.15.7:0/3590458164 conn(0x7ffb5c068bd0 msgr2=0x7ffb5c0c2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 shutdown_connections 2023-10-07T08:00:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.571+0000 7ffb63c3d700 1 -- 172.21.15.7:0/3590458164 wait complete. 2023-10-07T08:00:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-10-07T08:00:50.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T08:00:50.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.717+0000 7fc2dbde6700 1 Processor -- start 2023-10-07T08:00:50.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.717+0000 7fc2dbde6700 1 -- start start 2023-10-07T08:00:50.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.718+0000 7fc2dbde6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2d4159380 0x7fc2d40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.718+0000 7fc2dbde6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2d40d0250 0x7fc2d40d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.718+0000 7fc2dbde6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d40d0be0 con 0x7fc2d40b3b60 2023-10-07T08:00:50.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.718+0000 7fc2dbde6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d40b4820 con 0x7fc2d40d0250 2023-10-07T08:00:50.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.718+0000 7fc2dbde6700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d40b5000 con 0x7fc2d4159380 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.718+0000 7fc2d9381700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2d40d0250 0x7fc2d40d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.718+0000 7fc2d9381700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2d40d0250 0x7fc2d40d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47942/0 (socket says 172.21.15.7:47942) 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.718+0000 7fc2d9381700 1 -- 172.21.15.7:0/1953935696 learned_addr learned my addr 172.21.15.7:0/1953935696 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.719+0000 7fc2d9b82700 1 --2- 172.21.15.7:0/1953935696 >> v2:172.21.15.172:3300/0 conn(0x7fc2d4159380 0x7fc2d40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.719+0000 7fc2d9381700 1 -- 172.21.15.7:0/1953935696 >> v2:172.21.15.172:3300/0 conn(0x7fc2d4159380 msgr2=0x7fc2d40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.719+0000 7fc2d9381700 1 --2- 172.21.15.7:0/1953935696 >> v2:172.21.15.172:3300/0 conn(0x7fc2d4159380 0x7fc2d40b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.719+0000 7fc2d9381700 1 -- 172.21.15.7:0/1953935696 >> v1:172.21.15.172:6789/0 conn(0x7fc2d40b3b60 legacy=0x7fc2d40b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.719+0000 7fc2d9381700 1 -- 172.21.15.7:0/1953935696 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40b52e0 con 0x7fc2d40d0250 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.719+0000 7fc2d9381700 1 --2- 172.21.15.7:0/1953935696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40d0250 0x7fc2d40d06a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40b2e40 tx=0x7fc2d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b8ba6e04cf987c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.719+0000 7fc2d8b80700 1 -- 172.21.15.7:0/1953935696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000b820 con 0x7fc2d40d0250 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.719+0000 7fc2d8b80700 1 -- 172.21.15.7:0/1953935696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fc2d000b980 con 0x7fc2d40d0250 2023-10-07T08:00:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.719+0000 7fc2d8b80700 1 -- 172.21.15.7:0/1953935696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000bd20 con 0x7fc2d40d0250 2023-10-07T08:00:50.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.720+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1953935696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40d0250 msgr2=0x7fc2d40d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.720+0000 7fc2dbde6700 1 --2- 172.21.15.7:0/1953935696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40d0250 0x7fc2d40d06a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40b2e40 tx=0x7fc2d0009250 comp rx=0 tx=0).stop 2023-10-07T08:00:50.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.720+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1953935696 shutdown_connections 2023-10-07T08:00:50.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.720+0000 7fc2dbde6700 1 --2- 172.21.15.7:0/1953935696 >> v2:172.21.15.172:3300/0 conn(0x7fc2d4159380 0x7fc2d40b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.720+0000 7fc2dbde6700 1 --2- 172.21.15.7:0/1953935696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40d0250 0x7fc2d40d06a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.720+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1953935696 >> 172.21.15.7:0/1953935696 conn(0x7fc2d4067be0 msgr2=0x7fc2d4067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.720+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1953935696 shutdown_connections 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.720+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1953935696 wait complete. 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.721+0000 7fc2dbde6700 1 Processor -- start 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.721+0000 7fc2dbde6700 1 -- start start 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2dbde6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40b3b60 0x7fc2d40c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2dbde6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2d40d0250 0x7fc2d41514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2dbde6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2d4159380 0x7fc2d41563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2dbde6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d40c5900 con 0x7fc2d40b3b60 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2dbde6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d40c8250 con 0x7fc2d40d0250 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2dbde6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d40c8530 con 0x7fc2d4159380 2023-10-07T08:00:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2d9b82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40b3b60 0x7fc2d40c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2d9b82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40b3b60 0x7fc2d40c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47950/0 (socket says 172.21.15.7:47950) 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2d9b82700 1 -- 172.21.15.7:0/1841769015 learned_addr learned my addr 172.21.15.7:0/1841769015 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2d9381700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2d40d0250 0x7fc2d41514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2da383700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2d4159380 0x7fc2d41563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2d9b82700 1 -- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2d4159380 msgr2=0x7fc2d41563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2d9b82700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2d4159380 0x7fc2d41563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2d9b82700 1 -- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2d40d0250 msgr2=0x7fc2d41514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2d9b82700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2d40d0250 0x7fc2d41514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.722+0000 7fc2d9b82700 1 -- 172.21.15.7:0/1841769015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d00094f0 con 0x7fc2d40b3b60 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.723+0000 7fc2d9b82700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40b3b60 0x7fc2d40c9dc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800d890 tx=0x7fc2c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.723+0000 7fc2c5ffb700 1 -- 172.21.15.7:0/1841769015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80125f0 con 0x7fc2d40b3b60 2023-10-07T08:00:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.723+0000 7fc2c5ffb700 1 -- 172.21.15.7:0/1841769015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fc2c8012750 con 0x7fc2d40b3b60 2023-10-07T08:00:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.723+0000 7fc2c5ffb700 1 -- 172.21.15.7:0/1841769015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8012af0 con 0x7fc2d40b3b60 2023-10-07T08:00:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.723+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d40c8810 con 0x7fc2d40b3b60 2023-10-07T08:00:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.723+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d4156930 con 0x7fc2d40b3b60 2023-10-07T08:00:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.724+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d40c5750 con 0x7fc2d40b3b60 2023-10-07T08:00:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.724+0000 7fc2c5ffb700 1 -- 172.21.15.7:0/1841769015 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2c801e460 con 0x7fc2d40b3b60 2023-10-07T08:00:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.725+0000 7fc2c5ffb700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc2b8066f30 0x7fc2b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.725+0000 7fc2c5ffb700 1 -- 172.21.15.7:0/1841769015 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fc2c8084b80 con 0x7fc2d40b3b60 2023-10-07T08:00:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.725+0000 7fc2d9381700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc2b8066f30 0x7fc2b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.725+0000 7fc2d9381700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc2b8066f30 0x7fc2b80693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00090b0 tx=0x7fc2d0009250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:50.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.729+0000 7fc2c5ffb700 1 -- 172.21.15.7:0/1841769015 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2c8051de0 con 0x7fc2d40b3b60 2023-10-07T08:00:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.933+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2d4155c30 con 0x7fc2d40b3b60 2023-10-07T08:00:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.934+0000 7fc2c5ffb700 1 -- 172.21.15.7:0/1841769015 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15634 (secure 0 0 0) 0x7fc2c8055400 con 0x7fc2d40b3b60 2023-10-07T08:00:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:00:50.936 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T08:00:42.522687+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:59:48.327048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T07:59:48.760594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]},{"pgid":"7.39","mappings":[{"from":5,"to":0}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T08:00:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc2b8066f30 msgr2=0x7fc2b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc2b8066f30 0x7fc2b80693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00090b0 tx=0x7fc2d0009250 comp rx=0 tx=0).stop 2023-10-07T08:00:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40b3b60 msgr2=0x7fc2d40c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40b3b60 0x7fc2d40c9dc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800d890 tx=0x7fc2c8008040 comp rx=0 tx=0).stop 2023-10-07T08:00:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 shutdown_connections 2023-10-07T08:00:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc2b8066f30 0x7fc2b80693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2d4159380 0x7fc2d41563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d40b3b60 0x7fc2d40c9dc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 --2- 172.21.15.7:0/1841769015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2d40d0250 0x7fc2d41514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 >> 172.21.15.7:0/1841769015 conn(0x7fc2d4067be0 msgr2=0x7fc2d40b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 shutdown_connections 2023-10-07T08:00:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:50.937+0000 7fc2dbde6700 1 -- 172.21.15.7:0/1841769015 wait complete. 2023-10-07T08:00:50.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T07:57:44.081562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T07:59:48.327048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T07:59:48.760594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T08:00:50.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T08:00:51.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.086+0000 7faec8859700 1 Processor -- start 2023-10-07T08:00:51.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.086+0000 7faec8859700 1 -- start start 2023-10-07T08:00:51.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.087+0000 7faec8859700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faec4157300 0x7faec40afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec8859700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec40b18d0 con 0x7faec4156f30 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec8859700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec40b0fa0 con 0x7faec40affc0 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec8859700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec40b0440 con 0x7faec4157300 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faec4156f30 0x7faec40b1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44028/0 (socket says 172.21.15.7:44028) 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec259c700 1 -- 172.21.15.7:0/2124758501 learned_addr learned my addr 172.21.15.7:0/2124758501 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec1d9b700 1 --2- 172.21.15.7:0/2124758501 >> v2:172.21.15.172:3300/0 conn(0x7faec4157300 0x7faec40afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec159a700 1 -- 172.21.15.7:0/2124758501 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517039138 0 0) 0x7faec40b18d0 con 0x7faec4156f30 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec159a700 1 -- 172.21.15.7:0/2124758501 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeac003570 con 0x7faec4156f30 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec159a700 1 -- 172.21.15.7:0/2124758501 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522685931 0 0) 0x7faec40b0fa0 con 0x7faec40affc0 2023-10-07T08:00:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec159a700 1 -- 172.21.15.7:0/2124758501 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec40b18d0 con 0x7faec40affc0 2023-10-07T08:00:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec1d9b700 1 -- 172.21.15.7:0/2124758501 >> v1:172.21.15.172:6789/0 conn(0x7faec40affc0 legacy=0x7faec40b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec1d9b700 1 -- 172.21.15.7:0/2124758501 >> v1:172.21.15.7:6789/0 conn(0x7faec4156f30 legacy=0x7faec40b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec159a700 1 -- 172.21.15.7:0/2124758501 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 763510692 0 0) 0x7faeac003570 con 0x7faec4156f30 2023-10-07T08:00:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.088+0000 7faec1d9b700 1 -- 172.21.15.7:0/2124758501 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec40b1280 con 0x7faec4157300 2023-10-07T08:00:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.089+0000 7faec1d9b700 1 --2- 172.21.15.7:0/2124758501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 0x7faec40afa80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faeb80098e0 tx=0x7faeb80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75d735ecb0cf9ea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.089+0000 7faec159a700 1 -- 172.21.15.7:0/2124758501 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4001470 con 0x7faec4157300 2023-10-07T08:00:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.089+0000 7faec159a700 1 -- 172.21.15.7:0/2124758501 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7faeb8011420 con 0x7faec4157300 2023-10-07T08:00:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.089+0000 7faec159a700 1 -- 172.21.15.7:0/2124758501 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80117a0 con 0x7faec4157300 2023-10-07T08:00:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.089+0000 7faec8859700 1 -- 172.21.15.7:0/2124758501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 msgr2=0x7faec40afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.089+0000 7faec8859700 1 --2- 172.21.15.7:0/2124758501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 0x7faec40afa80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faeb80098e0 tx=0x7faeb80091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.089+0000 7faec8859700 1 -- 172.21.15.7:0/2124758501 shutdown_connections 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.089+0000 7faec8859700 1 --2- 172.21.15.7:0/2124758501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 0x7faec40afa80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.089+0000 7faec8859700 1 -- 172.21.15.7:0/2124758501 >> 172.21.15.7:0/2124758501 conn(0x7faec406d350 msgr2=0x7faec406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.090+0000 7faec8859700 1 -- 172.21.15.7:0/2124758501 shutdown_connections 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.090+0000 7faec8859700 1 -- 172.21.15.7:0/2124758501 wait complete. 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.091+0000 7faec8859700 1 Processor -- start 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.091+0000 7faec8859700 1 -- start start 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.091+0000 7faec8859700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec40affc0 0x7faec4156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.091+0000 7faec8859700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faec4156f30 0x7faec4154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.091+0000 7faec8859700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 0x7faec4150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.091+0000 7faec8859700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec4155610 con 0x7faec40affc0 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec8859700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40c30f0 con 0x7faec4157300 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec8859700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec40c3c00 con 0x7faec4156f30 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec40affc0 0x7faec4156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec40affc0 0x7faec4156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47966/0 (socket says 172.21.15.7:47966) 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec259c700 1 -- 172.21.15.7:0/480027426 learned_addr learned my addr 172.21.15.7:0/480027426 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec1d9b700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faec4156f30 0x7faec4154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec2d9d700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 0x7faec4150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec259c700 1 -- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faec4156f30 msgr2=0x7faec4154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec259c700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faec4156f30 0x7faec4154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec259c700 1 -- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 msgr2=0x7faec4150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec259c700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 0x7faec4150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec259c700 1 -- 172.21.15.7:0/480027426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8009420 con 0x7faec40affc0 2023-10-07T08:00:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec2d9d700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 0x7faec4150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec259c700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec40affc0 0x7faec4156370 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faeb400cb70 tx=0x7faeb4001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faeb27fc700 1 -- 172.21.15.7:0/480027426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4001d00 con 0x7faec40affc0 2023-10-07T08:00:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faeb27fc700 1 -- 172.21.15.7:0/480027426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7faeb4007360 con 0x7faec40affc0 2023-10-07T08:00:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec4155890 con 0x7faec40affc0 2023-10-07T08:00:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faeb27fc700 1 -- 172.21.15.7:0/480027426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4004830 con 0x7faec40affc0 2023-10-07T08:00:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.092+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec4155e50 con 0x7faec40affc0 2023-10-07T08:00:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.093+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec404c3c0 con 0x7faec40affc0 2023-10-07T08:00:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.094+0000 7faeb27fc700 1 -- 172.21.15.7:0/480027426 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faeb4004be0 con 0x7faec40affc0 2023-10-07T08:00:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.094+0000 7faeb27fc700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faea4066f30 0x7faea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.095+0000 7faeb27fc700 1 -- 172.21.15.7:0/480027426 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7faeb4083d30 con 0x7faec40affc0 2023-10-07T08:00:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.095+0000 7faec1d9b700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faea4066f30 0x7faea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.095+0000 7faec1d9b700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faea4066f30 0x7faea40693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faeb8003080 tx=0x7faeb80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.099+0000 7faeb27fc700 1 -- 172.21.15.7:0/480027426 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faeb4051070 con 0x7faec40affc0 2023-10-07T08:00:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.300+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faec40b1560 con 0x7faec40affc0 2023-10-07T08:00:51.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.301+0000 7faeb27fc700 1 -- 172.21.15.7:0/480027426 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7faeb4018020 con 0x7faec40affc0 2023-10-07T08:00:51.302 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-10-07T08:00:51.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faea4066f30 msgr2=0x7faea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faea4066f30 0x7faea40693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faeb8003080 tx=0x7faeb80091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec40affc0 msgr2=0x7faec4156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec40affc0 0x7faec4156370 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faeb400cb70 tx=0x7faeb4001ab0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 shutdown_connections 2023-10-07T08:00:51.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7faea4066f30 0x7faea40693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faec4157300 0x7faec4150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec40affc0 0x7faec4156370 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 --2- 172.21.15.7:0/480027426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faec4156f30 0x7faec4154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 >> 172.21.15.7:0/480027426 conn(0x7faec406d350 msgr2=0x7faec40ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:51.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 shutdown_connections 2023-10-07T08:00:51.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.303+0000 7faec8859700 1 -- 172.21.15.7:0/480027426 wait complete. 2023-10-07T08:00:51.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T08:00:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.452+0000 7f52ebe72700 1 Processor -- start 2023-10-07T08:00:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.452+0000 7f52ebe72700 1 -- start start 2023-10-07T08:00:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.453+0000 7f52ebe72700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52e415c920 0x7f52e40ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.453+0000 7f52ebe72700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52e415ccf0 0x7f52e406c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.453+0000 7f52ebe72700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e406c600 con 0x7f52e40c6d70 2023-10-07T08:00:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.453+0000 7f52ebe72700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e40cb3c0 con 0x7f52e415ccf0 2023-10-07T08:00:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.453+0000 7f52ebe72700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40cab00 con 0x7f52e415c920 2023-10-07T08:00:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.453+0000 7f52ea40f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f52e40c6d70 0x7f52e4068000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44032/0 (socket says 172.21.15.7:44032) 2023-10-07T08:00:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.453+0000 7f52ea40f700 1 -- 172.21.15.7:0/568426318 learned_addr learned my addr 172.21.15.7:0/568426318 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e9c0e700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52e415c920 0x7f52e40ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e940d700 1 --2- 172.21.15.7:0/568426318 >> v2:172.21.15.172:3300/0 conn(0x7f52e415ccf0 0x7f52e406c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117956320 0 0) 0x7f52dc001490 con 0x7f52e40c6d70 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52d0003570 con 0x7f52e40c6d70 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740762043 0 0) 0x7f52d0003570 con 0x7f52e40c6d70 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52e415c920 msgr2=0x7f52e40ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52e415c920 0x7f52e40ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 >> v2:172.21.15.172:3300/0 conn(0x7f52e415ccf0 msgr2=0x7f52e406c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 --2- 172.21.15.7:0/568426318 >> v2:172.21.15.172:3300/0 conn(0x7f52e415ccf0 0x7f52e406c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e406a7a0 con 0x7f52e40c6d70 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f52dc001980 con 0x7f52e40c6d70 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 >> v1:172.21.15.7:6789/0 conn(0x7f52e40c6d70 legacy=0x7f52e4068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d0003a60 0x7f52d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52e415c920 0x7f52d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0004d50 0x7f52d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40cb3c0 con 0x7f52d0003a60 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40cab00 con 0x7f52d0004d50 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d000bae0 con 0x7f52e415c920 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e9c0e700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d0003a60 0x7f52d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52e940d700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52e415c920 0x7f52d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.454+0000 7f52ea40f700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0004d50 0x7f52d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52e9c0e700 1 -- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52e415c920 msgr2=0x7f52d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52e9c0e700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52e415c920 0x7f52d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52e9c0e700 1 -- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0004d50 msgr2=0x7f52d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52e9c0e700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0004d50 0x7f52d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52e9c0e700 1 -- 172.21.15.7:0/568426318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d0003570 con 0x7f52d0003a60 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52ea40f700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0004d50 0x7f52d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52e9c0e700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d0003a60 0x7f52d0003e70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009bf0 tx=0x7f52e0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee6220771f0707c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0005a00 con 0x7f52d0003a60 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f52e0005b60 con 0x7f52d0003a60 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.455+0000 7f52e8c0c700 1 -- 172.21.15.7:0/568426318 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e000e710 con 0x7f52d0003a60 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.456+0000 7f52ebe72700 1 -- 172.21.15.7:0/568426318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d0003a60 msgr2=0x7f52d0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.456+0000 7f52ebe72700 1 --2- 172.21.15.7:0/568426318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d0003a60 0x7f52d0003e70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009bf0 tx=0x7f52e0009120 comp rx=0 tx=0).stop 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.456+0000 7f52ea40f700 1 -- 172.21.15.7:0/568426318 reap_dead start 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.456+0000 7f52ebe72700 1 -- 172.21.15.7:0/568426318 shutdown_connections 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.456+0000 7f52ebe72700 1 -- 172.21.15.7:0/568426318 >> 172.21.15.7:0/568426318 conn(0x7f52e40c9c30 msgr2=0x7f52e406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.456+0000 7f52ebe72700 1 -- 172.21.15.7:0/568426318 shutdown_connections 2023-10-07T08:00:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.456+0000 7f52ebe72700 1 -- 172.21.15.7:0/568426318 wait complete. 2023-10-07T08:00:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.457+0000 7f52ebe72700 1 Processor -- start 2023-10-07T08:00:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.457+0000 7f52ebe72700 1 -- start start 2023-10-07T08:00:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ebe72700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52d0003a60 0x7f52e40af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ebe72700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52e414d960 0x7f52e40c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ebe72700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e40b1760 0x7f52e40b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ebe72700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40b2140 con 0x7f52e40b1760 2023-10-07T08:00:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ebe72700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40c5310 con 0x7f52e414d960 2023-10-07T08:00:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ebe72700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e41590d0 con 0x7f52d0003a60 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ea40f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e40b1760 0x7f52e40b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ea40f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e40b1760 0x7f52e40b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47986/0 (socket says 172.21.15.7:47986) 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ea40f700 1 -- 172.21.15.7:0/2912156447 learned_addr learned my addr 172.21.15.7:0/2912156447 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52e9c0e700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52d0003a60 0x7f52e40af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52e940d700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52e414d960 0x7f52e40c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ea40f700 1 -- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52d0003a60 msgr2=0x7f52e40af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ea40f700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52d0003a60 0x7f52e40af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ea40f700 1 -- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52e414d960 msgr2=0x7f52e40c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ea40f700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52e414d960 0x7f52e40c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ea40f700 1 -- 172.21.15.7:0/2912156447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4159370 con 0x7f52e40b1760 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.458+0000 7f52ea40f700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e40b1760 0x7f52e40b1c00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f52e40b2730 tx=0x7f52e406a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.459+0000 7f52d5ffb700 1 -- 172.21.15.7:0/2912156447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc018070 con 0x7f52e40b1760 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.459+0000 7f52d5ffb700 1 -- 172.21.15.7:0/2912156447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f52dc007ad0 con 0x7f52e40b1760 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.459+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e40c47a0 con 0x7f52e40b1760 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.459+0000 7f52d5ffb700 1 -- 172.21.15.7:0/2912156447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00ed60 con 0x7f52e40b1760 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.459+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e40c4d50 con 0x7f52e40b1760 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.460+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e404c3c0 con 0x7f52e40b1760 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.460+0000 7f52d5ffb700 1 -- 172.21.15.7:0/2912156447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52dc013630 con 0x7f52e40b1760 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.461+0000 7f52d5ffb700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52c8066f60 0x7f52c8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.461+0000 7f52d5ffb700 1 -- 172.21.15.7:0/2912156447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f52dc084180 con 0x7f52e40b1760 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.461+0000 7f52e9c0e700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52c8066f60 0x7f52c8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.461+0000 7f52e9c0e700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52c8066f60 0x7f52c8069410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f52e000a010 tx=0x7f52e0009120 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.465+0000 7f52d5ffb700 1 -- 172.21.15.7:0/2912156447 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52dc0513e0 con 0x7f52e40b1760 2023-10-07T08:00:51.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.671+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52e406c600 con 0x7f52e40b1760 2023-10-07T08:00:51.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.672+0000 7f52d5ffb700 1 -- 172.21.15.7:0/2912156447 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52e406c600 con 0x7f52e40b1760 2023-10-07T08:00:51.673 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52c8066f60 msgr2=0x7f52c8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52c8066f60 0x7f52c8069410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f52e000a010 tx=0x7f52e0009120 comp rx=0 tx=0).stop 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e40b1760 msgr2=0x7f52e40b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e40b1760 0x7f52e40b1c00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f52e40b2730 tx=0x7f52e406a710 comp rx=0 tx=0).stop 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 shutdown_connections 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f52c8066f60 0x7f52c8069410 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e40b1760 0x7f52e40b1c00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52d0003a60 0x7f52e40af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 --2- 172.21.15.7:0/2912156447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52e414d960 0x7f52e40c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 >> 172.21.15.7:0/2912156447 conn(0x7f52e40c9c30 msgr2=0x7f52e40c88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 shutdown_connections 2023-10-07T08:00:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.674+0000 7f52ebe72700 1 -- 172.21.15.7:0/2912156447 wait complete. 2023-10-07T08:00:51.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T08:00:51.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.824+0000 7f652acf0700 1 Processor -- start 2023-10-07T08:00:51.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.824+0000 7f652acf0700 1 -- start start 2023-10-07T08:00:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.825+0000 7f652acf0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524068710 0x7f65240ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.825+0000 7f652acf0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65240cab00 con 0x7f652415ccf0 2023-10-07T08:00:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.825+0000 7f652acf0700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652406a7a0 con 0x7f652415c920 2023-10-07T08:00:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.825+0000 7f652acf0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65240cb3c0 con 0x7f6524068710 2023-10-07T08:00:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.825+0000 7f652928d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f652415ccf0 0x7f652406ac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44042/0 (socket says 172.21.15.7:44042) 2023-10-07T08:00:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.825+0000 7f652928d700 1 -- 172.21.15.7:0/4208979661 learned_addr learned my addr 172.21.15.7:0/4208979661 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.825+0000 7f6523fff700 1 --2- 172.21.15.7:0/4208979661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524068710 0x7f65240ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.825+0000 7f65237fe700 1 -- 172.21.15.7:0/4208979661 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268332675 0 0) 0x7f65240cab00 con 0x7f652415ccf0 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.825+0000 7f65237fe700 1 -- 172.21.15.7:0/4208979661 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6514003570 con 0x7f652415ccf0 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.826+0000 7f65237fe700 1 -- 172.21.15.7:0/4208979661 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009649263 0 0) 0x7f652406a7a0 con 0x7f652415c920 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.826+0000 7f65237fe700 1 -- 172.21.15.7:0/4208979661 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65240cab00 con 0x7f652415c920 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.826+0000 7f6523fff700 1 -- 172.21.15.7:0/4208979661 >> v1:172.21.15.172:6789/0 conn(0x7f652415c920 legacy=0x7f6524068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.826+0000 7f6523fff700 1 -- 172.21.15.7:0/4208979661 >> v1:172.21.15.7:6789/0 conn(0x7f652415ccf0 legacy=0x7f652406ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.826+0000 7f6523fff700 1 -- 172.21.15.7:0/4208979661 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65240cae00 con 0x7f6524068710 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.826+0000 7f6523fff700 1 --2- 172.21.15.7:0/4208979661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524068710 0x7f65240ca520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6510009910 tx=0x7f65100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=200111544c2da268 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.826+0000 7f65237fe700 1 -- 172.21.15.7:0/4208979661 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510011630 con 0x7f6524068710 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.826+0000 7f65237fe700 1 -- 172.21.15.7:0/4208979661 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f6510011790 con 0x7f6524068710 2023-10-07T08:00:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.826+0000 7f65237fe700 1 -- 172.21.15.7:0/4208979661 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510011b30 con 0x7f6524068710 2023-10-07T08:00:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.827+0000 7f652acf0700 1 -- 172.21.15.7:0/4208979661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524068710 msgr2=0x7f65240ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.827+0000 7f652acf0700 1 --2- 172.21.15.7:0/4208979661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524068710 0x7f65240ca520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6510009910 tx=0x7f65100091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.827+0000 7f652acf0700 1 -- 172.21.15.7:0/4208979661 shutdown_connections 2023-10-07T08:00:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.827+0000 7f652acf0700 1 --2- 172.21.15.7:0/4208979661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6524068710 0x7f65240ca520 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.827+0000 7f652acf0700 1 -- 172.21.15.7:0/4208979661 >> 172.21.15.7:0/4208979661 conn(0x7f65240c9c30 msgr2=0x7f652406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.827+0000 7f652acf0700 1 -- 172.21.15.7:0/4208979661 shutdown_connections 2023-10-07T08:00:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.827+0000 7f652acf0700 1 -- 172.21.15.7:0/4208979661 wait complete. 2023-10-07T08:00:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.828+0000 7f652acf0700 1 Processor -- start 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.828+0000 7f652acf0700 1 -- start start 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652acf0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6524068710 0x7f652414d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652acf0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f652415c920 0x7f652414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652acf0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f652415ccf0 0x7f652415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652acf0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652414f5d0 con 0x7f652415ccf0 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652acf0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652414db50 con 0x7f6524068710 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652acf0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652414ffa0 con 0x7f652415c920 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652928d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f652415ccf0 0x7f652415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652928d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f652415ccf0 0x7f652415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47988/0 (socket says 172.21.15.7:47988) 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652928d700 1 -- 172.21.15.7:0/1659063351 learned_addr learned my addr 172.21.15.7:0/1659063351 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f6528a8c700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6524068710 0x7f652414d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f6523fff700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f652415c920 0x7f652414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652928d700 1 -- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f652415c920 msgr2=0x7f652414fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652928d700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f652415c920 0x7f652414fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652928d700 1 -- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6524068710 msgr2=0x7f652414d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652928d700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6524068710 0x7f652414d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.829+0000 7f652928d700 1 -- 172.21.15.7:0/1659063351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510009420 con 0x7f652415ccf0 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.830+0000 7f652928d700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f652415ccf0 0x7f652415e4d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f651c00cc60 tx=0x7f651c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.830+0000 7f6520ff9700 1 -- 172.21.15.7:0/1659063351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00a8b0 con 0x7f652415ccf0 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.830+0000 7f6520ff9700 1 -- 172.21.15.7:0/1659063351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f651c00aa30 con 0x7f652415ccf0 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.830+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652414e9c0 con 0x7f652415ccf0 2023-10-07T08:00:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.830+0000 7f6520ff9700 1 -- 172.21.15.7:0/1659063351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00adf0 con 0x7f652415ccf0 2023-10-07T08:00:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.830+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652414ef80 con 0x7f652415ccf0 2023-10-07T08:00:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.831+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652404c3c0 con 0x7f652415ccf0 2023-10-07T08:00:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.831+0000 7f6520ff9700 1 -- 172.21.15.7:0/1659063351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f651c007300 con 0x7f652415ccf0 2023-10-07T08:00:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.832+0000 7f6520ff9700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6508066f30 0x7f65080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.832+0000 7f6520ff9700 1 -- 172.21.15.7:0/1659063351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f651c084930 con 0x7f652415ccf0 2023-10-07T08:00:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.832+0000 7f6528a8c700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6508066f30 0x7f65080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.833+0000 7f6528a8c700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6508066f30 0x7f65080693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6518003a40 tx=0x7f6518007d40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:51.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:51.836+0000 7f6520ff9700 1 -- 172.21.15.7:0/1659063351 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f651c055620 con 0x7f652415ccf0 2023-10-07T08:00:52.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.037+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f652406e4d0 con 0x7f652415ccf0 2023-10-07T08:00:52.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.037+0000 7f6520ff9700 1 -- 172.21.15.7:0/1659063351 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f651c0551b0 con 0x7f652415ccf0 2023-10-07T08:00:52.039 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-10-07T08:00:52.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6508066f30 msgr2=0x7f65080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6508066f30 0x7f65080693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6518003a40 tx=0x7f6518007d40 comp rx=0 tx=0).stop 2023-10-07T08:00:52.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f652415ccf0 msgr2=0x7f652415e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f652415ccf0 0x7f652415e4d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f651c00cc60 tx=0x7f651c001ae0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 shutdown_connections 2023-10-07T08:00:52.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f6508066f30 0x7f65080693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f652415ccf0 0x7f652415e4d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6524068710 0x7f652414d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 --2- 172.21.15.7:0/1659063351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f652415c920 0x7f652414fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 >> 172.21.15.7:0/1659063351 conn(0x7f65240c9c30 msgr2=0x7f6524151830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:52.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 shutdown_connections 2023-10-07T08:00:52.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.040+0000 7f652acf0700 1 -- 172.21.15.7:0/1659063351 wait complete. 2023-10-07T08:00:52.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T08:00:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.187+0000 7f5607bdb700 1 Processor -- start 2023-10-07T08:00:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.188+0000 7f5607bdb700 1 -- start start 2023-10-07T08:00:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.188+0000 7f5607bdb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 0x7f5600164110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.188+0000 7f5607bdb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560006bb70 con 0x7f56001592c0 2023-10-07T08:00:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5607bdb700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56000cda10 con 0x7f5600159690 2023-10-07T08:00:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5607bdb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560006af60 con 0x7f56000aaa50 2023-10-07T08:00:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5605977700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f56001592c0 0x7f560006a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44050/0 (socket says 172.21.15.7:44050) 2023-10-07T08:00:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5605977700 1 -- 172.21.15.7:0/3758389162 learned_addr learned my addr 172.21.15.7:0/3758389162 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5606178700 1 --2- 172.21.15.7:0/3758389162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 0x7f5600164110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5604975700 1 -- 172.21.15.7:0/3758389162 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663605572 0 0) 0x7f560006bb70 con 0x7f56001592c0 2023-10-07T08:00:52.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5604975700 1 -- 172.21.15.7:0/3758389162 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55ec003570 con 0x7f56001592c0 2023-10-07T08:00:52.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5604975700 1 -- 172.21.15.7:0/3758389162 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681960981 0 0) 0x7f56000cda10 con 0x7f5600159690 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5604975700 1 -- 172.21.15.7:0/3758389162 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f560006bb70 con 0x7f5600159690 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5604975700 1 -- 172.21.15.7:0/3758389162 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4276862094 0 0) 0x7f55ec003570 con 0x7f56001592c0 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5606178700 1 -- 172.21.15.7:0/3758389162 >> v1:172.21.15.172:6789/0 conn(0x7f5600159690 legacy=0x7f56000cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5606178700 1 -- 172.21.15.7:0/3758389162 >> v1:172.21.15.7:6789/0 conn(0x7f56001592c0 legacy=0x7f560006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.189+0000 7f5606178700 1 -- 172.21.15.7:0/3758389162 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560006b240 con 0x7f56000aaa50 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.190+0000 7f5606178700 1 --2- 172.21.15.7:0/3758389162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 0x7f5600164110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009e90 tx=0x7f55f8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2ce5266a3f49e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.190+0000 7f5604975700 1 -- 172.21.15.7:0/3758389162 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800e070 con 0x7f56000aaa50 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.190+0000 7f5604975700 1 -- 172.21.15.7:0/3758389162 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f55f800b870 con 0x7f56000aaa50 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.190+0000 7f5604975700 1 -- 172.21.15.7:0/3758389162 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc001470 con 0x7f56000aaa50 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.190+0000 7f5607bdb700 1 -- 172.21.15.7:0/3758389162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 msgr2=0x7f5600164110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.190+0000 7f5607bdb700 1 --2- 172.21.15.7:0/3758389162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 0x7f5600164110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009e90 tx=0x7f55f8009780 comp rx=0 tx=0).stop 2023-10-07T08:00:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.190+0000 7f5607bdb700 1 -- 172.21.15.7:0/3758389162 shutdown_connections 2023-10-07T08:00:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.190+0000 7f5607bdb700 1 --2- 172.21.15.7:0/3758389162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 0x7f5600164110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.190+0000 7f5607bdb700 1 -- 172.21.15.7:0/3758389162 >> 172.21.15.7:0/3758389162 conn(0x7f5600068bd0 msgr2=0x7f56000ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.191+0000 7f5607bdb700 1 -- 172.21.15.7:0/3758389162 shutdown_connections 2023-10-07T08:00:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.191+0000 7f5607bdb700 1 -- 172.21.15.7:0/3758389162 wait complete. 2023-10-07T08:00:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.192+0000 7f5607bdb700 1 Processor -- start 2023-10-07T08:00:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.192+0000 7f5607bdb700 1 -- start start 2023-10-07T08:00:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.192+0000 7f5607bdb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 0x7f5600156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.192+0000 7f5607bdb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56001592c0 0x7f56000ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.192+0000 7f5607bdb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5600159690 0x7f56000b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5607bdb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560016cbd0 con 0x7f5600159690 2023-10-07T08:00:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5607bdb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560016ceb0 con 0x7f56001592c0 2023-10-07T08:00:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5607bdb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560016d190 con 0x7f56000aaa50 2023-10-07T08:00:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5606178700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5600159690 0x7f56000b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5606178700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5600159690 0x7f56000b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47996/0 (socket says 172.21.15.7:47996) 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5606178700 1 -- 172.21.15.7:0/819612509 learned_addr learned my addr 172.21.15.7:0/819612509 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5605977700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 0x7f5600156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5605176700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56001592c0 0x7f56000ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5606178700 1 -- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 msgr2=0x7f5600156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5606178700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 0x7f5600156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5606178700 1 -- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56001592c0 msgr2=0x7f56000ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5606178700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56001592c0 0x7f56000ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5606178700 1 -- 172.21.15.7:0/819612509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f8009a20 con 0x7f5600159690 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f5606178700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5600159690 0x7f56000b1ae0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f55f800b610 tx=0x7f55f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f55f5ffb700 1 -- 172.21.15.7:0/819612509 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8012d20 con 0x7f5600159690 2023-10-07T08:00:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.193+0000 7f55f5ffb700 1 -- 172.21.15.7:0/819612509 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f55f8012e80 con 0x7f5600159690 2023-10-07T08:00:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.194+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560016d470 con 0x7f5600159690 2023-10-07T08:00:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.194+0000 7f55f5ffb700 1 -- 172.21.15.7:0/819612509 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f801e6a0 con 0x7f5600159690 2023-10-07T08:00:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.194+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56001572e0 con 0x7f5600159690 2023-10-07T08:00:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.195+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56000716c0 con 0x7f5600159690 2023-10-07T08:00:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.195+0000 7f55f5ffb700 1 -- 172.21.15.7:0/819612509 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55f801ead0 con 0x7f5600159690 2023-10-07T08:00:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.196+0000 7f55f5ffb700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55e4066f30 0x7f55e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.196+0000 7f55f5ffb700 1 -- 172.21.15.7:0/819612509 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f55f8086900 con 0x7f5600159690 2023-10-07T08:00:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.196+0000 7f5605977700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55e4066f30 0x7f55e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.196+0000 7f5605977700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55e4066f30 0x7f55e40693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0011e0 tx=0x7f55fc001a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.200+0000 7f55f5ffb700 1 -- 172.21.15.7:0/819612509 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55f8053c20 con 0x7f5600159690 2023-10-07T08:00:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.453+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5600157bd0 con 0x7f5600159690 2023-10-07T08:00:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.453+0000 7f55f5ffb700 1 -- 172.21.15.7:0/819612509 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4589 (secure 0 0 0) 0x7f55f802b090 con 0x7f5600159690 2023-10-07T08:00:52.454 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:00:52.455 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3622385241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3622385241},{"type":"v1","addr":"172.21.15.172:6837","nonce":3622385241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/899764672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":899764672},{"type":"v1","addr":"172.21.15.172:6836","nonce":899764672}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/318366060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":318366060},{"type":"v1","addr":"172.21.15.7:6833","nonce":318366060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:59:49.797329+0000","modified":"2023-10-07T08:00:46.532591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.7:6835/763397529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":763397529},{"type":"v1","addr":"172.21.15.7:6835","nonce":763397529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T08:00:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55e4066f30 msgr2=0x7f55e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55e4066f30 0x7f55e40693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0011e0 tx=0x7f55fc001a00 comp rx=0 tx=0).stop 2023-10-07T08:00:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5600159690 msgr2=0x7f56000b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5600159690 0x7f56000b1ae0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f55f800b610 tx=0x7f55f8009780 comp rx=0 tx=0).stop 2023-10-07T08:00:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 shutdown_connections 2023-10-07T08:00:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f55e4066f30 0x7f55e40693e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5600159690 0x7f56000b1ae0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56000aaa50 0x7f5600156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 --2- 172.21.15.7:0/819612509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56001592c0 0x7f56000ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 >> 172.21.15.7:0/819612509 conn(0x7f5600068bd0 msgr2=0x7f56000aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 shutdown_connections 2023-10-07T08:00:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.456+0000 7f5607bdb700 1 -- 172.21.15.7:0/819612509 wait complete. 2023-10-07T08:00:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-10-07T08:00:52.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T08:00:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.602+0000 7f16463f3700 1 Processor -- start 2023-10-07T08:00:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.602+0000 7f16463f3700 1 -- start start 2023-10-07T08:00:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.603+0000 7f16463f3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1640159690 0x7f16400cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.603+0000 7f16463f3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f16400a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.603+0000 7f16463f3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164006bb70 con 0x7f16401592c0 2023-10-07T08:00:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.603+0000 7f16463f3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16400cda10 con 0x7f1640159690 2023-10-07T08:00:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.603+0000 7f16463f3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164006af60 con 0x7f16400cb0c0 2023-10-07T08:00:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f163f7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1640159690 0x7f16400cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f1644990700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f16400a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f163f7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1640159690 0x7f16400cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:43080/0 (socket says 172.21.15.7:43080) 2023-10-07T08:00:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f163f7fe700 1 -- 172.21.15.7:0/626155905 learned_addr learned my addr 172.21.15.7:0/626155905 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f1644990700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f16400a9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:37962/0 (socket says 172.21.15.7:37962) 2023-10-07T08:00:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f163ffff700 1 --1- 172.21.15.7:0/626155905 >> v1:172.21.15.172:6789/0 conn(0x7f16401592c0 0x7f164006a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:51994/0 (socket says 172.21.15.7:51994) 2023-10-07T08:00:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f1644990700 1 -- 172.21.15.7:0/626155905 >> v2:172.21.15.172:3300/0 conn(0x7f1640159690 msgr2=0x7f16400cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f1644990700 1 --2- 172.21.15.7:0/626155905 >> v2:172.21.15.172:3300/0 conn(0x7f1640159690 0x7f16400cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f1644990700 1 -- 172.21.15.7:0/626155905 >> v1:172.21.15.172:6789/0 conn(0x7f16401592c0 legacy=0x7f164006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f1644990700 1 -- 172.21.15.7:0/626155905 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164006b240 con 0x7f16400cb0c0 2023-10-07T08:00:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f163f7fe700 1 --2- 172.21.15.7:0/626155905 >> v2:172.21.15.172:3300/0 conn(0x7f1640159690 0x7f16400cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:00:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f1644990700 1 --2- 172.21.15.7:0/626155905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f16400a9a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1638009e60 tx=0x7f1638009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f2dea54223616e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.604+0000 7f163effd700 1 -- 172.21.15.7:0/626155905 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800e070 con 0x7f16400cb0c0 2023-10-07T08:00:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f163effd700 1 -- 172.21.15.7:0/626155905 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f163800bb60 con 0x7f16400cb0c0 2023-10-07T08:00:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f163effd700 1 -- 172.21.15.7:0/626155905 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638012430 con 0x7f16400cb0c0 2023-10-07T08:00:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f16463f3700 1 -- 172.21.15.7:0/626155905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 msgr2=0x7f16400a9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f16463f3700 1 --2- 172.21.15.7:0/626155905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f16400a9a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1638009e60 tx=0x7f1638009780 comp rx=0 tx=0).stop 2023-10-07T08:00:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f16463f3700 1 -- 172.21.15.7:0/626155905 shutdown_connections 2023-10-07T08:00:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f16463f3700 1 --2- 172.21.15.7:0/626155905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f16400a9a40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f16463f3700 1 --2- 172.21.15.7:0/626155905 >> v2:172.21.15.172:3300/0 conn(0x7f1640159690 0x7f16400cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f16463f3700 1 -- 172.21.15.7:0/626155905 >> 172.21.15.7:0/626155905 conn(0x7f1640068bd0 msgr2=0x7f16400ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f16463f3700 1 -- 172.21.15.7:0/626155905 shutdown_connections 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.605+0000 7f16463f3700 1 -- 172.21.15.7:0/626155905 wait complete. 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.606+0000 7f16463f3700 1 Processor -- start 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.607+0000 7f16463f3700 1 -- start start 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.607+0000 7f16463f3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f1640154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.607+0000 7f16463f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401592c0 0x7f1640153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.607+0000 7f16463f3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1640159690 0x7f16400ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.607+0000 7f16463f3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16401513a0 con 0x7f16401592c0 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.607+0000 7f16463f3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16400ad560 con 0x7f1640159690 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.607+0000 7f16463f3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1640154750 con 0x7f16400cb0c0 2023-10-07T08:00:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401592c0 0x7f1640153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401592c0 0x7f1640153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48004/0 (socket says 172.21.15.7:48004) 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163f7fe700 1 -- 172.21.15.7:0/3310130692 learned_addr learned my addr 172.21.15.7:0/3310130692 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f1644990700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1640159690 0x7f16400ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163ffff700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f1640154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163f7fe700 1 -- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 msgr2=0x7f1640154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163f7fe700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f1640154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163f7fe700 1 -- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1640159690 msgr2=0x7f16400ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163f7fe700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1640159690 0x7f16400ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163f7fe700 1 -- 172.21.15.7:0/3310130692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340094f0 con 0x7f16401592c0 2023-10-07T08:00:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f1644990700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1640159690 0x7f16400ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:00:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f163f7fe700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401592c0 0x7f1640153e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f163400ec50 tx=0x7f1634009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f1627fff700 1 -- 172.21.15.7:0/3310130692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163400c850 con 0x7f16401592c0 2023-10-07T08:00:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f1627fff700 1 -- 172.21.15.7:0/3310130692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f163400c9b0 con 0x7f16401592c0 2023-10-07T08:00:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1638009a20 con 0x7f16401592c0 2023-10-07T08:00:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f1627fff700 1 -- 172.21.15.7:0/3310130692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163400cd50 con 0x7f16401592c0 2023-10-07T08:00:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.608+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16400ad1f0 con 0x7f16401592c0 2023-10-07T08:00:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.610+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16400ad6d0 con 0x7f16401592c0 2023-10-07T08:00:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.610+0000 7f1627fff700 1 -- 172.21.15.7:0/3310130692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1634013680 con 0x7f16401592c0 2023-10-07T08:00:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.610+0000 7f1627fff700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f162c066f30 0x7f162c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.610+0000 7f1627fff700 1 -- 172.21.15.7:0/3310130692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f16340846b0 con 0x7f16401592c0 2023-10-07T08:00:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.611+0000 7f163ffff700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f162c066f30 0x7f162c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.611+0000 7f163ffff700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f162c066f30 0x7f162c0693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1630004fd0 tx=0x7f1630007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.615+0000 7f1627fff700 1 -- 172.21.15.7:0/3310130692 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1634051890 con 0x7f16401592c0 2023-10-07T08:00:52.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.819+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16400716c0 con 0x7f16401592c0 2023-10-07T08:00:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.820+0000 7f1627fff700 1 -- 172.21.15.7:0/3310130692 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15634 (secure 0 0 0) 0x7f1634054eb0 con 0x7f16401592c0 2023-10-07T08:00:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:00:52.822 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T08:00:42.522687+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:59:48.327048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T07:59:48.760594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]},{"pgid":"7.39","mappings":[{"from":5,"to":0}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T08:00:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f162c066f30 msgr2=0x7f162c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f162c066f30 0x7f162c0693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1630004fd0 tx=0x7f1630007eb0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401592c0 msgr2=0x7f1640153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401592c0 0x7f1640153e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f163400ec50 tx=0x7f1634009250 comp rx=0 tx=0).stop 2023-10-07T08:00:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 shutdown_connections 2023-10-07T08:00:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f162c066f30 0x7f162c0693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1640159690 0x7f16400ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16400cb0c0 0x7f1640154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 --2- 172.21.15.7:0/3310130692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401592c0 0x7f1640153e00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 >> 172.21.15.7:0/3310130692 conn(0x7f1640068bd0 msgr2=0x7f16400aaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:52.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 shutdown_connections 2023-10-07T08:00:52.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.822+0000 7f16463f3700 1 -- 172.21.15.7:0/3310130692 wait complete. 2023-10-07T08:00:52.837 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-07T08:00:52.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T08:00:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.969+0000 7f3578e42700 1 Processor -- start 2023-10-07T08:00:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.969+0000 7f3578e42700 1 -- start start 2023-10-07T08:00:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.970+0000 7f3578e42700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.970+0000 7f3578e42700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f357406bd30 0x7f35740c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.970+0000 7f3578e42700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740c3b80 con 0x7f3574157470 2023-10-07T08:00:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.970+0000 7f3578e42700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35740c2f70 con 0x7f357406bd30 2023-10-07T08:00:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.970+0000 7f3578e42700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c3250 con 0x7f35741570a0 2023-10-07T08:00:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.970+0000 7f3572d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f357406bd30 0x7f35740c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.970+0000 7f3571d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3574157470 0x7f357406b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44066/0 (socket says 172.21.15.7:44066) 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.970+0000 7f3571d9b700 1 -- 172.21.15.7:0/92928452 learned_addr learned my addr 172.21.15.7:0/92928452 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.970+0000 7f357259c700 1 --2- 172.21.15.7:0/92928452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.971+0000 7f3572d9d700 1 -- 172.21.15.7:0/92928452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 msgr2=0x7f35740c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.971+0000 7f3572d9d700 1 --2- 172.21.15.7:0/92928452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.971+0000 7f3572d9d700 1 -- 172.21.15.7:0/92928452 >> v1:172.21.15.7:6789/0 conn(0x7f3574157470 legacy=0x7f357406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.971+0000 7f3572d9d700 1 -- 172.21.15.7:0/92928452 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740c3530 con 0x7f357406bd30 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.971+0000 7f357259c700 1 --2- 172.21.15.7:0/92928452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.971+0000 7f3572d9d700 1 --2- 172.21.15.7:0/92928452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f357406bd30 0x7f35740c4010 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f356c009e60 tx=0x7f356c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f98debb7c2c0f6a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.971+0000 7f357159a700 1 -- 172.21.15.7:0/92928452 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00e070 con 0x7f357406bd30 2023-10-07T08:00:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.971+0000 7f357159a700 1 -- 172.21.15.7:0/92928452 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f356c00bc30 con 0x7f357406bd30 2023-10-07T08:00:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.971+0000 7f357159a700 1 -- 172.21.15.7:0/92928452 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c012430 con 0x7f357406bd30 2023-10-07T08:00:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.972+0000 7f3578e42700 1 -- 172.21.15.7:0/92928452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f357406bd30 msgr2=0x7f35740c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.972+0000 7f3578e42700 1 --2- 172.21.15.7:0/92928452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f357406bd30 0x7f35740c4010 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f356c009e60 tx=0x7f356c009780 comp rx=0 tx=0).stop 2023-10-07T08:00:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.972+0000 7f3578e42700 1 -- 172.21.15.7:0/92928452 shutdown_connections 2023-10-07T08:00:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.972+0000 7f3578e42700 1 --2- 172.21.15.7:0/92928452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f357406bd30 0x7f35740c4010 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.972+0000 7f3578e42700 1 --2- 172.21.15.7:0/92928452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.972+0000 7f3578e42700 1 -- 172.21.15.7:0/92928452 >> 172.21.15.7:0/92928452 conn(0x7f3574067ac0 msgr2=0x7f35740ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.972+0000 7f3578e42700 1 -- 172.21.15.7:0/92928452 shutdown_connections 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.972+0000 7f3578e42700 1 -- 172.21.15.7:0/92928452 wait complete. 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.973+0000 7f3578e42700 1 Processor -- start 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.973+0000 7f3578e42700 1 -- start start 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3578e42700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f357406bd30 0x7f357413e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3578e42700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3578e42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574157470 0x7f3574155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3578e42700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357413f6a0 con 0x7f3574157470 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3578e42700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357413fb30 con 0x7f357406bd30 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3578e42700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357413ea90 con 0x7f35741570a0 2023-10-07T08:00:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3572d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574157470 0x7f3574155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3572d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574157470 0x7f3574155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48026/0 (socket says 172.21.15.7:48026) 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3572d9d700 1 -- 172.21.15.7:0/341321825 learned_addr learned my addr 172.21.15.7:0/341321825 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f357259c700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f357406bd30 0x7f357413e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3571d9b700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3572d9d700 1 -- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 msgr2=0x7f35740cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3572d9d700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3572d9d700 1 -- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f357406bd30 msgr2=0x7f357413e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3572d9d700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f357406bd30 0x7f357413e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3572d9d700 1 -- 172.21.15.7:0/341321825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c009a20 con 0x7f3574157470 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3571d9b700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.974+0000 7f3572d9d700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574157470 0x7f3574155a40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f35740cbc20 tx=0x7f356c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.975+0000 7f356a7fc700 1 -- 172.21.15.7:0/341321825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c027470 con 0x7f3574157470 2023-10-07T08:00:52.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.975+0000 7f356a7fc700 1 -- 172.21.15.7:0/341321825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f356c0275f0 con 0x7f3574157470 2023-10-07T08:00:52.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.975+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357413ed70 con 0x7f3574157470 2023-10-07T08:00:52.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.975+0000 7f356a7fc700 1 -- 172.21.15.7:0/341321825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c0279b0 con 0x7f3574157470 2023-10-07T08:00:52.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.975+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357413f330 con 0x7f3574157470 2023-10-07T08:00:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.976+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740716c0 con 0x7f3574157470 2023-10-07T08:00:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.976+0000 7f356a7fc700 1 -- 172.21.15.7:0/341321825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f356c024070 con 0x7f3574157470 2023-10-07T08:00:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.977+0000 7f356a7fc700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3554066f30 0x7f35540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.977+0000 7f356a7fc700 1 -- 172.21.15.7:0/341321825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f356c086c10 con 0x7f3574157470 2023-10-07T08:00:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.977+0000 7f357259c700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3554066f30 0x7f35540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.977+0000 7f357259c700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3554066f30 0x7f35540693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3564006f00 tx=0x7f3564008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:52.981+0000 7f356a7fc700 1 -- 172.21.15.7:0/341321825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f356c053dd0 con 0x7f3574157470 2023-10-07T08:00:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.226+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35740cea10 con 0x7f3574157470 2023-10-07T08:00:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.227+0000 7f356a7fc700 1 -- 172.21.15.7:0/341321825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4589 (secure 0 0 0) 0x7f356c02c090 con 0x7f3574157470 2023-10-07T08:00:53.228 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:00:53.228 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3622385241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3622385241},{"type":"v1","addr":"172.21.15.172:6837","nonce":3622385241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/899764672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":899764672},{"type":"v1","addr":"172.21.15.172:6836","nonce":899764672}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/318366060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":318366060},{"type":"v1","addr":"172.21.15.7:6833","nonce":318366060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:59:49.797329+0000","modified":"2023-10-07T08:00:46.532591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.7:6835/763397529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":763397529},{"type":"v1","addr":"172.21.15.7:6835","nonce":763397529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T08:00:53.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3554066f30 msgr2=0x7f35540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3554066f30 0x7f35540693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3564006f00 tx=0x7f3564008040 comp rx=0 tx=0).stop 2023-10-07T08:00:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574157470 msgr2=0x7f3574155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574157470 0x7f3574155a40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f35740cbc20 tx=0x7f356c009780 comp rx=0 tx=0).stop 2023-10-07T08:00:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 shutdown_connections 2023-10-07T08:00:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3554066f30 0x7f35540693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574157470 0x7f3574155a40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f357406bd30 0x7f357413e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 --2- 172.21.15.7:0/341321825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35741570a0 0x7f35740cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 >> 172.21.15.7:0/341321825 conn(0x7f3574067ac0 msgr2=0x7f3574139930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 shutdown_connections 2023-10-07T08:00:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.229+0000 7f3578e42700 1 -- 172.21.15.7:0/341321825 wait complete. 2023-10-07T08:00:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-10-07T08:00:53.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T08:00:53.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.377+0000 7fd97a038700 1 Processor -- start 2023-10-07T08:00:53.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.377+0000 7fd97a038700 1 -- start start 2023-10-07T08:00:53.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.377+0000 7fd97a038700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9740c9af0 0x7fd97415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd97a038700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd9740c9ec0 0x7fd97406e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd97a038700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97415a330 con 0x7fd974063230 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd97a038700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9740b03a0 con 0x7fd9740c9af0 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd97a038700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97415a610 con 0x7fd9740c9ec0 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd9737fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9740c9af0 0x7fd97415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd972ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd974063230 0x7fd97406db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44074/0 (socket says 172.21.15.7:44074) 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd972ffd700 1 -- 172.21.15.7:0/860400060 learned_addr learned my addr 172.21.15.7:0/860400060 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd973fff700 1 --2- 172.21.15.7:0/860400060 >> v2:172.21.15.172:3300/0 conn(0x7fd9740c9ec0 0x7fd97406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd9737fe700 1 -- 172.21.15.7:0/860400060 >> v2:172.21.15.172:3300/0 conn(0x7fd9740c9ec0 msgr2=0x7fd97406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd9737fe700 1 --2- 172.21.15.7:0/860400060 >> v2:172.21.15.172:3300/0 conn(0x7fd9740c9ec0 0x7fd97406e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd9737fe700 1 -- 172.21.15.7:0/860400060 >> v1:172.21.15.7:6789/0 conn(0x7fd974063230 legacy=0x7fd97406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd9737fe700 1 -- 172.21.15.7:0/860400060 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97415a8f0 con 0x7fd9740c9af0 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd9727fc700 1 -- 172.21.15.7:0/860400060 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419616692 0 0) 0x7fd97415a330 con 0x7fd974063230 2023-10-07T08:00:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd973fff700 1 --2- 172.21.15.7:0/860400060 >> v2:172.21.15.172:3300/0 conn(0x7fd9740c9ec0 0x7fd97406e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd9737fe700 1 --2- 172.21.15.7:0/860400060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9af0 0x7fd97415d8f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd9740ae6e0 tx=0x7fd9640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf253efeb03d682 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.378+0000 7fd9727fc700 1 -- 172.21.15.7:0/860400060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96400b6b0 con 0x7fd9740c9af0 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd9727fc700 1 -- 172.21.15.7:0/860400060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fd96400b810 con 0x7fd9740c9af0 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd9727fc700 1 -- 172.21.15.7:0/860400060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96400bbb0 con 0x7fd9740c9af0 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd97a038700 1 -- 172.21.15.7:0/860400060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9af0 msgr2=0x7fd97415d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd97a038700 1 --2- 172.21.15.7:0/860400060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9af0 0x7fd97415d8f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd9740ae6e0 tx=0x7fd9640091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd97a038700 1 -- 172.21.15.7:0/860400060 shutdown_connections 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd97a038700 1 --2- 172.21.15.7:0/860400060 >> v2:172.21.15.172:3300/0 conn(0x7fd9740c9ec0 0x7fd97406e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd97a038700 1 --2- 172.21.15.7:0/860400060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9af0 0x7fd97415d8f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd97a038700 1 -- 172.21.15.7:0/860400060 >> 172.21.15.7:0/860400060 conn(0x7fd9740cd1a0 msgr2=0x7fd9740c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd97a038700 1 -- 172.21.15.7:0/860400060 shutdown_connections 2023-10-07T08:00:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.379+0000 7fd97a038700 1 -- 172.21.15.7:0/860400060 wait complete. 2023-10-07T08:00:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.381+0000 7fd97a038700 1 Processor -- start 2023-10-07T08:00:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.381+0000 7fd97a038700 1 -- start start 2023-10-07T08:00:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd97a038700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd974063230 0x7fd9740c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd97a038700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9740c9af0 0x7fd974147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd97a038700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9ec0 0x7fd9740a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd97a038700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740cb5c0 con 0x7fd9740c9ec0 2023-10-07T08:00:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd97a038700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740ca9b0 con 0x7fd9740c9af0 2023-10-07T08:00:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd97a038700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740cac90 con 0x7fd974063230 2023-10-07T08:00:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd973fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9ec0 0x7fd9740a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd973fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9ec0 0x7fd9740a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48042/0 (socket says 172.21.15.7:48042) 2023-10-07T08:00:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd973fff700 1 -- 172.21.15.7:0/4240102117 learned_addr learned my addr 172.21.15.7:0/4240102117 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd972ffd700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9740c9af0 0x7fd974147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd9737fe700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd974063230 0x7fd9740c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd973fff700 1 -- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd974063230 msgr2=0x7fd9740c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd973fff700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd974063230 0x7fd9740c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd973fff700 1 -- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9740c9af0 msgr2=0x7fd974147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd973fff700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9740c9af0 0x7fd974147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd973fff700 1 -- 172.21.15.7:0/4240102117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd964009420 con 0x7fd9740c9ec0 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd973fff700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9ec0 0x7fd9740a5480 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00d180 tx=0x7fd96c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.382+0000 7fd95b7fe700 1 -- 172.21.15.7:0/4240102117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00dcc0 con 0x7fd9740c9ec0 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.383+0000 7fd95b7fe700 1 -- 172.21.15.7:0/4240102117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fd96c00f040 con 0x7fd9740c9ec0 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.383+0000 7fd95b7fe700 1 -- 172.21.15.7:0/4240102117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c015690 con 0x7fd9740c9ec0 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.383+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9740caf70 con 0x7fd9740c9ec0 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.383+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9740c89b0 con 0x7fd9740c9ec0 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.384+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9740cb770 con 0x7fd9740c9ec0 2023-10-07T08:00:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.384+0000 7fd95b7fe700 1 -- 172.21.15.7:0/4240102117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd96c01e3b0 con 0x7fd9740c9ec0 2023-10-07T08:00:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.385+0000 7fd95b7fe700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd960066f30 0x7fd9600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.385+0000 7fd95b7fe700 1 -- 172.21.15.7:0/4240102117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fd96c084a80 con 0x7fd9740c9ec0 2023-10-07T08:00:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.385+0000 7fd9737fe700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd960066f30 0x7fd9600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.385+0000 7fd9737fe700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd960066f30 0x7fd9600693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd9740c8c50 tx=0x7fd9640091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.389+0000 7fd95b7fe700 1 -- 172.21.15.7:0/4240102117 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd96c051ce0 con 0x7fd9740c9ec0 2023-10-07T08:00:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.638+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9740716c0 con 0x7fd9740c9ec0 2023-10-07T08:00:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.639+0000 7fd95b7fe700 1 -- 172.21.15.7:0/4240102117 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4589 (secure 0 0 0) 0x7fd96c01a090 con 0x7fd9740c9ec0 2023-10-07T08:00:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:00:53.640 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3622385241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3622385241},{"type":"v1","addr":"172.21.15.172:6837","nonce":3622385241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/899764672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":899764672},{"type":"v1","addr":"172.21.15.172:6836","nonce":899764672}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/318366060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":318366060},{"type":"v1","addr":"172.21.15.7:6833","nonce":318366060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T07:59:49.797329+0000","modified":"2023-10-07T08:00:46.532591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.7:6835/763397529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":763397529},{"type":"v1","addr":"172.21.15.7:6835","nonce":763397529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T08:00:53.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd960066f30 msgr2=0x7fd9600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd960066f30 0x7fd9600693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd9740c8c50 tx=0x7fd9640091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9ec0 msgr2=0x7fd9740a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9ec0 0x7fd9740a5480 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00d180 tx=0x7fd96c009a50 comp rx=0 tx=0).stop 2023-10-07T08:00:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 shutdown_connections 2023-10-07T08:00:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fd960066f30 0x7fd9600693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9740c9ec0 0x7fd9740a5480 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd974063230 0x7fd9740c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 --2- 172.21.15.7:0/4240102117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9740c9af0 0x7fd974147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 >> 172.21.15.7:0/4240102117 conn(0x7fd9740cd1a0 msgr2=0x7fd974158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:53.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 shutdown_connections 2023-10-07T08:00:53.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.641+0000 7fd97a038700 1 -- 172.21.15.7:0/4240102117 wait complete. 2023-10-07T08:00:53.644 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-10-07T08:00:53.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T08:00:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.788+0000 7f621235a700 1 Processor -- start 2023-10-07T08:00:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.788+0000 7f621235a700 1 -- start start 2023-10-07T08:00:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f621235a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f620c155f20 0x7f620c0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f621235a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f620c06e9d0 0x7f620c06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f621235a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620c06b700 con 0x7f620c1562f0 2023-10-07T08:00:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f621235a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c06aaf0 con 0x7f620c06e9d0 2023-10-07T08:00:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f621235a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c06add0 con 0x7f620c155f20 2023-10-07T08:00:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f62108f7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f620c06e9d0 0x7f620c06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f62108f7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f620c06e9d0 0x7f620c06a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48048/0 (socket says 172.21.15.7:48048) 2023-10-07T08:00:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f62108f7700 1 -- 172.21.15.7:0/3949049222 learned_addr learned my addr 172.21.15.7:0/3949049222 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f620bfff700 1 --2- 172.21.15.7:0/3949049222 >> v2:172.21.15.172:3300/0 conn(0x7f620c155f20 0x7f620c0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f62108f7700 1 -- 172.21.15.7:0/3949049222 >> v2:172.21.15.172:3300/0 conn(0x7f620c155f20 msgr2=0x7f620c0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f62108f7700 1 --2- 172.21.15.7:0/3949049222 >> v2:172.21.15.172:3300/0 conn(0x7f620c155f20 0x7f620c0c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f62108f7700 1 -- 172.21.15.7:0/3949049222 >> v1:172.21.15.172:6789/0 conn(0x7f620c1562f0 legacy=0x7f620c06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f62108f7700 1 -- 172.21.15.7:0/3949049222 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c06b0b0 con 0x7f620c06e9d0 2023-10-07T08:00:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.789+0000 7f62108f7700 1 --2- 172.21.15.7:0/3949049222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 0x7f620c06a3f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f620c070c00 tx=0x7f6204009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858684f36eed5d65 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f620affd700 1 -- 172.21.15.7:0/3949049222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620400e070 con 0x7f620c06e9d0 2023-10-07T08:00:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f620affd700 1 -- 172.21.15.7:0/3949049222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f620400ba20 con 0x7f620c06e9d0 2023-10-07T08:00:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f620affd700 1 -- 172.21.15.7:0/3949049222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62040092e0 con 0x7f620c06e9d0 2023-10-07T08:00:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f621235a700 1 -- 172.21.15.7:0/3949049222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 msgr2=0x7f620c06a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f621235a700 1 --2- 172.21.15.7:0/3949049222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 0x7f620c06a3f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f620c070c00 tx=0x7f6204009840 comp rx=0 tx=0).stop 2023-10-07T08:00:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f621235a700 1 -- 172.21.15.7:0/3949049222 shutdown_connections 2023-10-07T08:00:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f621235a700 1 --2- 172.21.15.7:0/3949049222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 0x7f620c06a3f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f621235a700 1 --2- 172.21.15.7:0/3949049222 >> v2:172.21.15.172:3300/0 conn(0x7f620c155f20 0x7f620c0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f621235a700 1 -- 172.21.15.7:0/3949049222 >> 172.21.15.7:0/3949049222 conn(0x7f620c068bd0 msgr2=0x7f620c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f621235a700 1 -- 172.21.15.7:0/3949049222 shutdown_connections 2023-10-07T08:00:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.790+0000 7f621235a700 1 -- 172.21.15.7:0/3949049222 wait complete. 2023-10-07T08:00:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.791+0000 7f621235a700 1 Processor -- start 2023-10-07T08:00:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.792+0000 7f621235a700 1 -- start start 2023-10-07T08:00:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.792+0000 7f621235a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 0x7f620c153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.792+0000 7f621235a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c155f20 0x7f620c0c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.792+0000 7f621235a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1562f0 0x7f620c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f621235a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0c6370 con 0x7f620c06e9d0 2023-10-07T08:00:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f621235a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c153640 con 0x7f620c1562f0 2023-10-07T08:00:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f621235a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0c5760 con 0x7f620c155f20 2023-10-07T08:00:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 0x7f620c153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 0x7f620c153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48056/0 (socket says 172.21.15.7:48056) 2023-10-07T08:00:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620bfff700 1 -- 172.21.15.7:0/1742284503 learned_addr learned my addr 172.21.15.7:0/1742284503 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620b7fe700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c155f20 0x7f620c0c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f62108f7700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1562f0 0x7f620c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620bfff700 1 -- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c155f20 msgr2=0x7f620c0c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620bfff700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c155f20 0x7f620c0c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620bfff700 1 -- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1562f0 msgr2=0x7f620c0a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620bfff700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1562f0 0x7f620c0a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620bfff700 1 -- 172.21.15.7:0/1742284503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6204009ae0 con 0x7f620c06e9d0 2023-10-07T08:00:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f62108f7700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1562f0 0x7f620c0a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f620bfff700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 0x7f620c153100 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00ec80 tx=0x7f61fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f61f3fff700 1 -- 172.21.15.7:0/1742284503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00c9d0 con 0x7f620c06e9d0 2023-10-07T08:00:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f61f3fff700 1 -- 172.21.15.7:0/1742284503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f61fc00cb30 con 0x7f620c06e9d0 2023-10-07T08:00:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c0c5a40 con 0x7f620c06e9d0 2023-10-07T08:00:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f61f3fff700 1 -- 172.21.15.7:0/1742284503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc013470 con 0x7f620c06e9d0 2023-10-07T08:00:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.793+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c0c6000 con 0x7f620c06e9d0 2023-10-07T08:00:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.794+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c0716c0 con 0x7f620c06e9d0 2023-10-07T08:00:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.795+0000 7f61f3fff700 1 -- 172.21.15.7:0/1742284503 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61fc013820 con 0x7f620c06e9d0 2023-10-07T08:00:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.795+0000 7f61f3fff700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f61f8066f30 0x7f61f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.795+0000 7f61f3fff700 1 -- 172.21.15.7:0/1742284503 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f61fc0849a0 con 0x7f620c06e9d0 2023-10-07T08:00:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.796+0000 7f620b7fe700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f61f8066f30 0x7f61f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:53.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.796+0000 7f620b7fe700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f61f8066f30 0x7f61f80693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f620c0c3330 tx=0x7f6200009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:53.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:53.800+0000 7f61f3fff700 1 -- 172.21.15.7:0/1742284503 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61fc051c00 con 0x7f620c06e9d0 2023-10-07T08:00:54.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.004+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f620c06f1f0 con 0x7f620c06e9d0 2023-10-07T08:00:54.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.005+0000 7f61f3fff700 1 -- 172.21.15.7:0/1742284503 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15634 (secure 0 0 0) 0x7f61fc055220 con 0x7f620c06e9d0 2023-10-07T08:00:54.006 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:00:54.006 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T08:00:42.522687+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T07:59:48.327048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T07:59:48.760594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]},{"pgid":"7.39","mappings":[{"from":5,"to":0}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T08:00:54.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f61f8066f30 msgr2=0x7f61f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f61f8066f30 0x7f61f80693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f620c0c3330 tx=0x7f6200009040 comp rx=0 tx=0).stop 2023-10-07T08:00:54.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 msgr2=0x7f620c153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 0x7f620c153100 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00ec80 tx=0x7f61fc0091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 shutdown_connections 2023-10-07T08:00:54.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f61f8066f30 0x7f61f80693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f620c1562f0 0x7f620c0a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f620c06e9d0 0x7f620c153100 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 --2- 172.21.15.7:0/1742284503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f620c155f20 0x7f620c0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 >> 172.21.15.7:0/1742284503 conn(0x7f620c068bd0 msgr2=0x7f620c154ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:54.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 shutdown_connections 2023-10-07T08:00:54.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.007+0000 7f621235a700 1 -- 172.21.15.7:0/1742284503 wait complete. 2023-10-07T08:00:54.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-07T08:00:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.155+0000 7fcfea5a2700 1 Processor -- start 2023-10-07T08:00:54.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.156+0000 7fcfea5a2700 1 -- start start 2023-10-07T08:00:54.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.156+0000 7fcfea5a2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4158f10 0x7fcfe41592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.156+0000 7fcfea5a2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfe4066ff0 0x7fcfe4067420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.156+0000 7fcfea5a2700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfe40abb90 0x7fcfe40abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.156+0000 7fcfea5a2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe4067960 con 0x7fcfe4066ff0 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.156+0000 7fcfea5a2700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe4159940 con 0x7fcfe40abb90 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.156+0000 7fcfea5a2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4159c20 con 0x7fcfe4158f10 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe8d9f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfe4066ff0 0x7fcfe4067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe8d9f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfe4066ff0 0x7fcfe4067420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48072/0 (socket says 172.21.15.7:48072) 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe8d9f700 1 -- 172.21.15.7:0/1957839552 learned_addr learned my addr 172.21.15.7:0/1957839552 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe9da1700 1 --2- 172.21.15.7:0/1957839552 >> v2:172.21.15.172:3300/0 conn(0x7fcfe40abb90 0x7fcfe40abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe95a0700 1 --2- 172.21.15.7:0/1957839552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4158f10 0x7fcfe41592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe8d9f700 1 -- 172.21.15.7:0/1957839552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4158f10 msgr2=0x7fcfe41592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe8d9f700 1 --2- 172.21.15.7:0/1957839552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4158f10 0x7fcfe41592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe8d9f700 1 -- 172.21.15.7:0/1957839552 >> v2:172.21.15.172:3300/0 conn(0x7fcfe40abb90 msgr2=0x7fcfe40abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe8d9f700 1 --2- 172.21.15.7:0/1957839552 >> v2:172.21.15.172:3300/0 conn(0x7fcfe40abb90 0x7fcfe40abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe8d9f700 1 -- 172.21.15.7:0/1957839552 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe4159f00 con 0x7fcfe4066ff0 2023-10-07T08:00:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfe8d9f700 1 --2- 172.21.15.7:0/1957839552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4066ff0 0x7fcfe4067420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcfe406e960 tx=0x7fcfd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23278293618360 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfd7fff700 1 -- 172.21.15.7:0/1957839552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800b640 con 0x7fcfe4066ff0 2023-10-07T08:00:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.157+0000 7fcfd7fff700 1 -- 172.21.15.7:0/1957839552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fcfd800b7a0 con 0x7fcfe4066ff0 2023-10-07T08:00:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfd7fff700 1 -- 172.21.15.7:0/1957839552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800bb60 con 0x7fcfe4066ff0 2023-10-07T08:00:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1957839552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4066ff0 msgr2=0x7fcfe4067420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1957839552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4066ff0 0x7fcfe4067420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcfe406e960 tx=0x7fcfd8009250 comp rx=0 tx=0).stop 2023-10-07T08:00:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1957839552 shutdown_connections 2023-10-07T08:00:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1957839552 >> v2:172.21.15.172:3300/0 conn(0x7fcfe40abb90 0x7fcfe40abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1957839552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4158f10 0x7fcfe41592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1957839552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4066ff0 0x7fcfe4067420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1957839552 >> 172.21.15.7:0/1957839552 conn(0x7fcfe4068100 msgr2=0x7fcfe4068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:54.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1957839552 shutdown_connections 2023-10-07T08:00:54.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.158+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1957839552 wait complete. 2023-10-07T08:00:54.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.159+0000 7fcfea5a2700 1 Processor -- start 2023-10-07T08:00:54.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.159+0000 7fcfea5a2700 1 -- start start 2023-10-07T08:00:54.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.160+0000 7fcfea5a2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4066ff0 0x7fcfe40abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.160+0000 7fcfea5a2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfe40abb90 0x7fcfe4149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.160+0000 7fcfea5a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4158f10 0x7fcfe40b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.160+0000 7fcfea5a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe414a050 con 0x7fcfe4158f10 2023-10-07T08:00:54.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.160+0000 7fcfea5a2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe414bfc0 con 0x7fcfe40abb90 2023-10-07T08:00:54.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.160+0000 7fcfea5a2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe40ac6e0 con 0x7fcfe4066ff0 2023-10-07T08:00:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.160+0000 7fcfe9da1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4158f10 0x7fcfe40b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe9da1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4158f10 0x7fcfe40b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48082/0 (socket says 172.21.15.7:48082) 2023-10-07T08:00:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe9da1700 1 -- 172.21.15.7:0/1968523111 learned_addr learned my addr 172.21.15.7:0/1968523111 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe95a0700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4066ff0 0x7fcfe40abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe8d9f700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfe40abb90 0x7fcfe4149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe9da1700 1 -- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4066ff0 msgr2=0x7fcfe40abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe9da1700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4066ff0 0x7fcfe40abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe9da1700 1 -- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfe40abb90 msgr2=0x7fcfe4149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe9da1700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfe40abb90 0x7fcfe4149b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe9da1700 1 -- 172.21.15.7:0/1968523111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80094f0 con 0x7fcfe4158f10 2023-10-07T08:00:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe95a0700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4066ff0 0x7fcfe40abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:00:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe8d9f700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfe40abb90 0x7fcfe4149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:00:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfe9da1700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4158f10 0x7fcfe40b1c60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009e90 tx=0x7fcfdc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfd57fa700 1 -- 172.21.15.7:0/1968523111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00cc20 con 0x7fcfe4158f10 2023-10-07T08:00:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfd57fa700 1 -- 172.21.15.7:0/1968523111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fcfdc00f040 con 0x7fcfe4158f10 2023-10-07T08:00:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe40ac9c0 con 0x7fcfe4158f10 2023-10-07T08:00:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfd57fa700 1 -- 172.21.15.7:0/1968523111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc014480 con 0x7fcfe4158f10 2023-10-07T08:00:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.161+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe40acf80 con 0x7fcfe4158f10 2023-10-07T08:00:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.162+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe404c3c0 con 0x7fcfe4158f10 2023-10-07T08:00:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.163+0000 7fcfd57fa700 1 -- 172.21.15.7:0/1968523111 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfdc014830 con 0x7fcfe4158f10 2023-10-07T08:00:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.163+0000 7fcfd57fa700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfc8066f30 0x7fcfc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.163+0000 7fcfd57fa700 1 -- 172.21.15.7:0/1968523111 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fcfdc0852f0 con 0x7fcfe4158f10 2023-10-07T08:00:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.163+0000 7fcfe95a0700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfc8066f30 0x7fcfc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.164+0000 7fcfe95a0700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfc8066f30 0x7fcfc80693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00093f0 tx=0x7fcfe0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.168+0000 7fcfd57fa700 1 -- 172.21.15.7:0/1968523111 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfdc055fe0 con 0x7fcfe4158f10 2023-10-07T08:00:54.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.421+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcfe406eed0 con 0x7fcfe4158f10 2023-10-07T08:00:54.423 INFO:tasks.ceph.mon.a.smithi007.stderr:2023-10-07T08:00:54.422+0000 7feeb1c30700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T08:00:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.426+0000 7fcfd57fa700 1 -- 172.21.15.7:0/1968523111 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcfdc055b70 con 0x7fcfe4158f10 2023-10-07T08:00:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.428+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfc8066f30 msgr2=0x7fcfc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.428+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfc8066f30 0x7fcfc80693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00093f0 tx=0x7fcfe0008040 comp rx=0 tx=0).stop 2023-10-07T08:00:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.428+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4158f10 msgr2=0x7fcfe40b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.428+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4158f10 0x7fcfe40b1c60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009e90 tx=0x7fcfdc00b040 comp rx=0 tx=0).stop 2023-10-07T08:00:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.429+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 shutdown_connections 2023-10-07T08:00:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.429+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fcfc8066f30 0x7fcfc80693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.429+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe4158f10 0x7fcfe40b1c60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.429+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfe4066ff0 0x7fcfe40abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.429+0000 7fcfea5a2700 1 --2- 172.21.15.7:0/1968523111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfe40abb90 0x7fcfe4149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.429+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 >> 172.21.15.7:0/1968523111 conn(0x7fcfe4068100 msgr2=0x7fcfe40b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.429+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 shutdown_connections 2023-10-07T08:00:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.429+0000 7fcfea5a2700 1 -- 172.21.15.7:0/1968523111 wait complete. 2023-10-07T08:00:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-07T08:00:54.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-07T08:00:54.468 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2023-10-07T08:00:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.579+0000 7f29c0448700 1 Processor -- start 2023-10-07T08:00:54.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.580+0000 7f29c0448700 1 -- start start 2023-10-07T08:00:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29c0448700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29b813f840 0x7f29b806e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29c0448700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b80c7300 con 0x7f29b813fc10 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29c0448700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b806e580 con 0x7f29b80c6bc0 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29c0448700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b806e840 con 0x7f29b813f840 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29be1e4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29b813f840 0x7f29b806e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29bd9e3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f29b813fc10 0x7f29b80c64b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44082/0 (socket says 172.21.15.7:44082) 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29bd9e3700 1 -- 172.21.15.7:0/3190512221 learned_addr learned my addr 172.21.15.7:0/3190512221 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29bd1e2700 1 -- 172.21.15.7:0/3190512221 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998229078 0 0) 0x7f29b80c7300 con 0x7f29b813fc10 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29bd1e2700 1 -- 172.21.15.7:0/3190512221 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a4003570 con 0x7f29b813fc10 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29be1e4700 1 -- 172.21.15.7:0/3190512221 >> v1:172.21.15.172:6789/0 conn(0x7f29b80c6bc0 legacy=0x7f29b80c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29be1e4700 1 -- 172.21.15.7:0/3190512221 >> v1:172.21.15.7:6789/0 conn(0x7f29b813fc10 legacy=0x7f29b80c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29be1e4700 1 -- 172.21.15.7:0/3190512221 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b806eb20 con 0x7f29b813f840 2023-10-07T08:00:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.581+0000 7f29be1e4700 1 --2- 172.21.15.7:0/3190512221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b813f840 0x7f29b806e040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f29b400bb70 tx=0x7f29b400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496db541dd787cff server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.582+0000 7f29bd1e2700 1 -- 172.21.15.7:0/3190512221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4005c10 con 0x7f29b813f840 2023-10-07T08:00:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.582+0000 7f29bd1e2700 1 -- 172.21.15.7:0/3190512221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f29b4005d70 con 0x7f29b813f840 2023-10-07T08:00:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.582+0000 7f29bd1e2700 1 -- 172.21.15.7:0/3190512221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b400db40 con 0x7f29b813f840 2023-10-07T08:00:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.582+0000 7f29c0448700 1 -- 172.21.15.7:0/3190512221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b813f840 msgr2=0x7f29b806e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.582+0000 7f29c0448700 1 --2- 172.21.15.7:0/3190512221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b813f840 0x7f29b806e040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f29b400bb70 tx=0x7f29b400b3e0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.582+0000 7f29c0448700 1 -- 172.21.15.7:0/3190512221 shutdown_connections 2023-10-07T08:00:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.582+0000 7f29c0448700 1 --2- 172.21.15.7:0/3190512221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b813f840 0x7f29b806e040 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.582+0000 7f29c0448700 1 -- 172.21.15.7:0/3190512221 >> 172.21.15.7:0/3190512221 conn(0x7f29b80cbe70 msgr2=0x7f29b80cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.582+0000 7f29c0448700 1 -- 172.21.15.7:0/3190512221 shutdown_connections 2023-10-07T08:00:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.583+0000 7f29c0448700 1 -- 172.21.15.7:0/3190512221 wait complete. 2023-10-07T08:00:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.584+0000 7f29c0448700 1 Processor -- start 2023-10-07T08:00:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.584+0000 7f29c0448700 1 -- start start 2023-10-07T08:00:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.584+0000 7f29c0448700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b80c6bc0 0x7f29b80c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.584+0000 7f29c0448700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b813f840 0x7f29b80c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.584+0000 7f29c0448700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b813fc10 0x7f29b8169620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.584+0000 7f29c0448700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b813b150 con 0x7f29b80c6bc0 2023-10-07T08:00:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.584+0000 7f29c0448700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c5860 con 0x7f29b813fc10 2023-10-07T08:00:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.584+0000 7f29c0448700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b816aee0 con 0x7f29b813f840 2023-10-07T08:00:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be1e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b80c6bc0 0x7f29b80c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be1e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b80c6bc0 0x7f29b80c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48128/0 (socket says 172.21.15.7:48128) 2023-10-07T08:00:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be1e4700 1 -- 172.21.15.7:0/1344777674 learned_addr learned my addr 172.21.15.7:0/1344777674 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be9e5700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b813fc10 0x7f29b8169620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29bd9e3700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b813f840 0x7f29b80c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be1e4700 1 -- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b813f840 msgr2=0x7f29b80c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be1e4700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b813f840 0x7f29b80c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be1e4700 1 -- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b813fc10 msgr2=0x7f29b8169620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be1e4700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b813fc10 0x7f29b8169620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be1e4700 1 -- 172.21.15.7:0/1344777674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b400b660 con 0x7f29b80c6bc0 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be9e5700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b813fc10 0x7f29b8169620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29be1e4700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b80c6bc0 0x7f29b80c47d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f29b4010f80 tx=0x7f29b400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29ae7fc700 1 -- 172.21.15.7:0/1344777674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b401e070 con 0x7f29b80c6bc0 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29ae7fc700 1 -- 172.21.15.7:0/1344777674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f29b400d930 con 0x7f29b80c6bc0 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29ae7fc700 1 -- 172.21.15.7:0/1344777674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4019680 con 0x7f29b80c6bc0 2023-10-07T08:00:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b80c5580 con 0x7f29b80c6bc0 2023-10-07T08:00:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.585+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b816ab70 con 0x7f29b80c6bc0 2023-10-07T08:00:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.586+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b804c3c0 con 0x7f29b80c6bc0 2023-10-07T08:00:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.587+0000 7f29ae7fc700 1 -- 172.21.15.7:0/1344777674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29b4019a70 con 0x7f29b80c6bc0 2023-10-07T08:00:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.587+0000 7f29ae7fc700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f299c066f30 0x7f299c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.588+0000 7f29ae7fc700 1 -- 172.21.15.7:0/1344777674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5936+0+0 (secure 0 0 0) 0x7f29b4014030 con 0x7f29b80c6bc0 2023-10-07T08:00:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.588+0000 7f29bd9e3700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f299c066f30 0x7f299c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.588+0000 7f29bd9e3700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f299c066f30 0x7f299c0693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f29b816a7a0 tx=0x7f29a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.592+0000 7f29ae7fc700 1 -- 172.21.15.7:0/1344777674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29b40530f0 con 0x7f29b80c6bc0 2023-10-07T08:00:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:54.846+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f29b80cb960 con 0x7f29b80c6bc0 2023-10-07T08:00:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.427+0000 7f29ae7fc700 1 -- 172.21.15.7:0/1344777674 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f29b4056710 con 0x7f29b80c6bc0 2023-10-07T08:00:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f299c066f30 msgr2=0x7f299c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f299c066f30 0x7f299c0693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f29b816a7a0 tx=0x7f29a8009040 comp rx=0 tx=0).stop 2023-10-07T08:00:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b80c6bc0 msgr2=0x7f29b80c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b80c6bc0 0x7f29b80c47d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f29b4010f80 tx=0x7f29b400b3e0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 shutdown_connections 2023-10-07T08:00:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f299c066f30 0x7f299c0693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b813fc10 0x7f29b8169620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b80c6bc0 0x7f29b80c47d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 --2- 172.21.15.7:0/1344777674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b813f840 0x7f29b80c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 >> 172.21.15.7:0/1344777674 conn(0x7f29b80cbe70 msgr2=0x7f29b806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 shutdown_connections 2023-10-07T08:00:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.431+0000 7f29c0448700 1 -- 172.21.15.7:0/1344777674 wait complete. 2023-10-07T08:00:55.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-07T08:00:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.634+0000 7f114ae84700 1 Processor -- start 2023-10-07T08:00:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.635+0000 7f114ae84700 1 -- start start 2023-10-07T08:00:55.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.635+0000 7f114ae84700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1144156040 0x7f11440ad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:55.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.635+0000 7f114ae84700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11440ad7d0 0x7f11440c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:55.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.635+0000 7f114ae84700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114406e9f0 con 0x7f11440c87e0 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.635+0000 7f114ae84700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440c7e50 con 0x7f1144156040 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.635+0000 7f114ae84700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440af990 con 0x7f11440ad7d0 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f1148c20700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1144156040 0x7f11440ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f1148c20700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1144156040 0x7f11440ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48136/0 (socket says 172.21.15.7:48136) 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f1148c20700 1 -- 172.21.15.7:0/3326768765 learned_addr learned my addr 172.21.15.7:0/3326768765 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f1143fff700 1 --2- 172.21.15.7:0/3326768765 >> v2:172.21.15.172:3300/0 conn(0x7f11440ad7d0 0x7f11440c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f1148c20700 1 -- 172.21.15.7:0/3326768765 >> v2:172.21.15.172:3300/0 conn(0x7f11440ad7d0 msgr2=0x7f11440c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f1148c20700 1 --2- 172.21.15.7:0/3326768765 >> v2:172.21.15.172:3300/0 conn(0x7f11440ad7d0 0x7f11440c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f1148c20700 1 -- 172.21.15.7:0/3326768765 >> v1:172.21.15.172:6789/0 conn(0x7f11440c87e0 legacy=0x7f114406a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f1148c20700 1 -- 172.21.15.7:0/3326768765 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440c8c20 con 0x7f1144156040 2023-10-07T08:00:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f1148c20700 1 --2- 172.21.15.7:0/3326768765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1144156040 0x7f11440ad290 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f114406b9f0 tx=0x7f1134009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fdd5d7b5cf18b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f11437fe700 1 -- 172.21.15.7:0/3326768765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134009e70 con 0x7f1144156040 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f11437fe700 1 -- 172.21.15.7:0/3326768765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f113400b860 con 0x7f1144156040 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.636+0000 7f11437fe700 1 -- 172.21.15.7:0/3326768765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113400bc00 con 0x7f1144156040 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.637+0000 7f114ae84700 1 -- 172.21.15.7:0/3326768765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1144156040 msgr2=0x7f11440ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.637+0000 7f114ae84700 1 --2- 172.21.15.7:0/3326768765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1144156040 0x7f11440ad290 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f114406b9f0 tx=0x7f1134009250 comp rx=0 tx=0).stop 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.637+0000 7f114ae84700 1 -- 172.21.15.7:0/3326768765 shutdown_connections 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.637+0000 7f114ae84700 1 --2- 172.21.15.7:0/3326768765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1144156040 0x7f11440ad290 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.637+0000 7f114ae84700 1 --2- 172.21.15.7:0/3326768765 >> v2:172.21.15.172:3300/0 conn(0x7f11440ad7d0 0x7f11440c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.637+0000 7f114ae84700 1 -- 172.21.15.7:0/3326768765 >> 172.21.15.7:0/3326768765 conn(0x7f1144068100 msgr2=0x7f1144068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.637+0000 7f114ae84700 1 -- 172.21.15.7:0/3326768765 shutdown_connections 2023-10-07T08:00:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.637+0000 7f114ae84700 1 -- 172.21.15.7:0/3326768765 wait complete. 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.638+0000 7f114ae84700 1 Processor -- start 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f114ae84700 1 -- start start 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f114ae84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11440ad7d0 0x7f11440caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f114ae84700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11440c87e0 0x7f1144156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f114ae84700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1144156040 0x7f114414c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f114ae84700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440cbf00 con 0x7f11440ad7d0 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f114ae84700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440cb2f0 con 0x7f11440c87e0 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f114ae84700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11440cb5d0 con 0x7f1144156040 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f1148c20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11440ad7d0 0x7f11440caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f1148c20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11440ad7d0 0x7f11440caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48138/0 (socket says 172.21.15.7:48138) 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.639+0000 7f1148c20700 1 -- 172.21.15.7:0/2651152820 learned_addr learned my addr 172.21.15.7:0/2651152820 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1143fff700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11440c87e0 0x7f1144156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1149421700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1144156040 0x7f114414c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1148c20700 1 -- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1144156040 msgr2=0x7f114414c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1148c20700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1144156040 0x7f114414c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1148c20700 1 -- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11440c87e0 msgr2=0x7f1144156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1148c20700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11440c87e0 0x7f1144156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1148c20700 1 -- 172.21.15.7:0/2651152820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340094f0 con 0x7f11440ad7d0 2023-10-07T08:00:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1149421700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1144156040 0x7f114414c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1148c20700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11440ad7d0 0x7f11440caaa0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f11340090b0 tx=0x7f1134009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1140ff9700 1 -- 172.21.15.7:0/2651152820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113400bbe0 con 0x7f11440ad7d0 2023-10-07T08:00:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1140ff9700 1 -- 172.21.15.7:0/2651152820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f113401a450 con 0x7f11440ad7d0 2023-10-07T08:00:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11440cb8b0 con 0x7f11440ad7d0 2023-10-07T08:00:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f1140ff9700 1 -- 172.21.15.7:0/2651152820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113401a7f0 con 0x7f11440ad7d0 2023-10-07T08:00:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.640+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11440cafe0 con 0x7f11440ad7d0 2023-10-07T08:00:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.641+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11440716c0 con 0x7f11440ad7d0 2023-10-07T08:00:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.642+0000 7f1140ff9700 1 -- 172.21.15.7:0/2651152820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f113401ab60 con 0x7f11440ad7d0 2023-10-07T08:00:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.642+0000 7f1140ff9700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1124066f30 0x7f11240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.642+0000 7f1140ff9700 1 -- 172.21.15.7:0/2651152820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5936+0+0 (secure 0 0 0) 0x7f1134085bd0 con 0x7f11440ad7d0 2023-10-07T08:00:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.643+0000 7f1143fff700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1124066f30 0x7f11240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.643+0000 7f1143fff700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1124066f30 0x7f11240693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1144161fd0 tx=0x7f112c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.647+0000 7f1140ff9700 1 -- 172.21.15.7:0/2651152820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1134052d60 con 0x7f11440ad7d0 2023-10-07T08:00:55.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:55.846+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f114406e3c0 con 0x7f11440ad7d0 2023-10-07T08:00:56.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.430+0000 7f1140ff9700 1 -- 172.21.15.7:0/2651152820 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7f11340200d0 con 0x7f11440ad7d0 2023-10-07T08:00:56.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.434+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1124066f30 msgr2=0x7f11240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:56.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.434+0000 7f114ae84700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1124066f30 0x7f11240693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1144161fd0 tx=0x7f112c0091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.434+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11440ad7d0 msgr2=0x7f11440caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:56.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.434+0000 7f114ae84700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11440ad7d0 0x7f11440caaa0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f11340090b0 tx=0x7f1134009250 comp rx=0 tx=0).stop 2023-10-07T08:00:56.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.435+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 shutdown_connections 2023-10-07T08:00:56.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.435+0000 7f114ae84700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1124066f30 0x7f11240693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.435+0000 7f114ae84700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1144156040 0x7f114414c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.435+0000 7f114ae84700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11440ad7d0 0x7f11440caaa0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.435+0000 7f114ae84700 1 --2- 172.21.15.7:0/2651152820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11440c87e0 0x7f1144156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.435+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 >> 172.21.15.7:0/2651152820 conn(0x7f1144068100 msgr2=0x7f11440c98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.436+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 shutdown_connections 2023-10-07T08:00:56.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.436+0000 7f114ae84700 1 -- 172.21.15.7:0/2651152820 wait complete. 2023-10-07T08:00:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2023-10-07T08:00:56.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-07T08:00:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.593+0000 7f1a1ddf3700 1 Processor -- start 2023-10-07T08:00:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.593+0000 7f1a1ddf3700 1 -- start start 2023-10-07T08:00:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.593+0000 7f1a1ddf3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a18151750 0x7f1a1815b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.593+0000 7f1a1ddf3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1a181585f0 0x7f1a1806b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.593+0000 7f1a1ddf3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 0x7f1a1806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a1ddf3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a180c8f40 con 0x7f1a18151750 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a1ddf3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a18157890 con 0x7f1a181585f0 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a1ddf3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a180c8c60 con 0x7f1a18151b20 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a177fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a18151750 0x7f1a1815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a177fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a18151750 0x7f1a1815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48154/0 (socket says 172.21.15.7:48154) 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a177fe700 1 -- 172.21.15.7:0/2091405716 learned_addr learned my addr 172.21.15.7:0/2091405716 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a17fff700 1 --2- 172.21.15.7:0/2091405716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 0x7f1a1806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a16ffd700 1 --2- 172.21.15.7:0/2091405716 >> v2:172.21.15.172:3300/0 conn(0x7f1a181585f0 0x7f1a1806b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a177fe700 1 -- 172.21.15.7:0/2091405716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 msgr2=0x7f1a1806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a177fe700 1 --2- 172.21.15.7:0/2091405716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 0x7f1a1806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a177fe700 1 -- 172.21.15.7:0/2091405716 >> v2:172.21.15.172:3300/0 conn(0x7f1a181585f0 msgr2=0x7f1a1806b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a177fe700 1 --2- 172.21.15.7:0/2091405716 >> v2:172.21.15.172:3300/0 conn(0x7f1a181585f0 0x7f1a1806b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a177fe700 1 -- 172.21.15.7:0/2091405716 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a18157b10 con 0x7f1a18151750 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a16ffd700 1 --2- 172.21.15.7:0/2091405716 >> v2:172.21.15.172:3300/0 conn(0x7f1a181585f0 0x7f1a1806b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a177fe700 1 --2- 172.21.15.7:0/2091405716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18151750 0x7f1a1815b850 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1a1806a4e0 tx=0x7f1a08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec67e6e28132699a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a167fc700 1 -- 172.21.15.7:0/2091405716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800b860 con 0x7f1a18151750 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a167fc700 1 -- 172.21.15.7:0/2091405716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f1a0800b9c0 con 0x7f1a18151750 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.594+0000 7f1a167fc700 1 -- 172.21.15.7:0/2091405716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800bd60 con 0x7f1a18151750 2023-10-07T08:00:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.595+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/2091405716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18151750 msgr2=0x7f1a1815b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.595+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/2091405716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18151750 0x7f1a1815b850 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1a1806a4e0 tx=0x7f1a08009250 comp rx=0 tx=0).stop 2023-10-07T08:00:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.595+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/2091405716 shutdown_connections 2023-10-07T08:00:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.595+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/2091405716 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 0x7f1a1806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.595+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/2091405716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18151750 0x7f1a1815b850 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.595+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/2091405716 >> v2:172.21.15.172:3300/0 conn(0x7f1a181585f0 0x7f1a1806b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.595+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/2091405716 >> 172.21.15.7:0/2091405716 conn(0x7f1a18067a70 msgr2=0x7f1a180ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:56.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.595+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/2091405716 shutdown_connections 2023-10-07T08:00:56.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.595+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/2091405716 wait complete. 2023-10-07T08:00:56.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.596+0000 7f1a1ddf3700 1 Processor -- start 2023-10-07T08:00:56.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a1ddf3700 1 -- start start 2023-10-07T08:00:56.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a1ddf3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a18151750 0x7f1a180b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:56.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a1ddf3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 0x7f1a18168b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:56.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a1ddf3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a181585f0 0x7f1a180b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:56.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a1ddf3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a180cd4c0 con 0x7f1a181585f0 2023-10-07T08:00:56.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a1ddf3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a180cd7a0 con 0x7f1a18151750 2023-10-07T08:00:56.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a1ddf3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a180cda80 con 0x7f1a18151b20 2023-10-07T08:00:56.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a181585f0 0x7f1a180b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:56.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a181585f0 0x7f1a180b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48156/0 (socket says 172.21.15.7:48156) 2023-10-07T08:00:56.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.597+0000 7f1a17fff700 1 -- 172.21.15.7:0/1314041375 learned_addr learned my addr 172.21.15.7:0/1314041375 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:56.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a177fe700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a18151750 0x7f1a180b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:56.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a16ffd700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 0x7f1a18168b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:56.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a17fff700 1 -- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 msgr2=0x7f1a18168b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:56.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a17fff700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 0x7f1a18168b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a17fff700 1 -- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a18151750 msgr2=0x7f1a180b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:56.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a17fff700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a18151750 0x7f1a180b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:56.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a17fff700 1 -- 172.21.15.7:0/1314041375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080094f0 con 0x7f1a181585f0 2023-10-07T08:00:56.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a17fff700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a181585f0 0x7f1a180b2250 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009fa0 tx=0x7f1a10009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:56.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f19ff7fe700 1 -- 172.21.15.7:0/1314041375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000ebc0 con 0x7f1a181585f0 2023-10-07T08:00:56.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f19ff7fe700 1 -- 172.21.15.7:0/1314041375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f1a1000ed20 con 0x7f1a181585f0 2023-10-07T08:00:56.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a180681a0 con 0x7f1a181585f0 2023-10-07T08:00:56.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f19ff7fe700 1 -- 172.21.15.7:0/1314041375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10014670 con 0x7f1a181585f0 2023-10-07T08:00:56.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.598+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a180b1f30 con 0x7f1a181585f0 2023-10-07T08:00:56.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.599+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a180716c0 con 0x7f1a181585f0 2023-10-07T08:00:56.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.600+0000 7f19ff7fe700 1 -- 172.21.15.7:0/1314041375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a100149e0 con 0x7f1a181585f0 2023-10-07T08:00:56.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.600+0000 7f19ff7fe700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1a04066f30 0x7f1a040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:56.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.600+0000 7f19ff7fe700 1 -- 172.21.15.7:0/1314041375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5498+0+0 (secure 0 0 0) 0x7f1a10084b10 con 0x7f1a181585f0 2023-10-07T08:00:56.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.601+0000 7f1a177fe700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1a04066f30 0x7f1a040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:56.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.601+0000 7f1a177fe700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1a04066f30 0x7f1a040693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1a080090b0 tx=0x7f1a08009250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.605+0000 7f19ff7fe700 1 -- 172.21.15.7:0/1314041375 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a10051ed0 con 0x7f1a181585f0 2023-10-07T08:00:56.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:56.810+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1a180ce9b0 con 0x7f1a181585f0 2023-10-07T08:00:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.435+0000 7f19ff7fe700 1 -- 172.21.15.7:0/1314041375 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1a100554f0 con 0x7f1a181585f0 2023-10-07T08:00:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.438+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1a04066f30 msgr2=0x7f1a040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.438+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1a04066f30 0x7f1a040693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1a080090b0 tx=0x7f1a08009250 comp rx=0 tx=0).stop 2023-10-07T08:00:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.438+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a181585f0 msgr2=0x7f1a180b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.438+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a181585f0 0x7f1a180b2250 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009fa0 tx=0x7f1a10009900 comp rx=0 tx=0).stop 2023-10-07T08:00:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.439+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 shutdown_connections 2023-10-07T08:00:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.439+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1a04066f30 0x7f1a040693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.439+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a181585f0 0x7f1a180b2250 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.439+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a18151750 0x7f1a180b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.439+0000 7f1a1ddf3700 1 --2- 172.21.15.7:0/1314041375 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a18151b20 0x7f1a18168b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.439+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 >> 172.21.15.7:0/1314041375 conn(0x7f1a18067a70 msgr2=0x7f1a1806ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:57.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.439+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 shutdown_connections 2023-10-07T08:00:57.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.439+0000 7f1a1ddf3700 1 -- 172.21.15.7:0/1314041375 wait complete. 2023-10-07T08:00:57.441 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2023-10-07T08:00:57.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_completed_requests 2023-10-07T08:00:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.594+0000 7f30f5968700 1 Processor -- start 2023-10-07T08:00:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.594+0000 7f30f5968700 1 -- start start 2023-10-07T08:00:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.595+0000 7f30f5968700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f015d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.595+0000 7f30f5968700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f00687f0 con 0x7f30f006b610 2023-10-07T08:00:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.595+0000 7f30f5968700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f006db90 con 0x7f30f0069c90 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.595+0000 7f30f5968700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f0159820 con 0x7f30f015d3b0 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.595+0000 7f30ee7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f30f006b610 0x7f30f006cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44084/0 (socket says 172.21.15.7:44084) 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.595+0000 7f30ee7fc700 1 -- 172.21.15.7:0/3745363067 learned_addr learned my addr 172.21.15.7:0/3745363067 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30eeffd700 1 --2- 172.21.15.7:0/3745363067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f015d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30edffb700 1 -- 172.21.15.7:0/3745363067 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109536455 0 0) 0x7f30f00687f0 con 0x7f30f006b610 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30edffb700 1 -- 172.21.15.7:0/3745363067 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30dc003570 con 0x7f30f006b610 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30edffb700 1 -- 172.21.15.7:0/3745363067 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310978306 0 0) 0x7f30f006db90 con 0x7f30f0069c90 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30edffb700 1 -- 172.21.15.7:0/3745363067 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f00687f0 con 0x7f30f0069c90 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30edffb700 1 -- 172.21.15.7:0/3745363067 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373219118 0 0) 0x7f30dc003570 con 0x7f30f006b610 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30eeffd700 1 -- 172.21.15.7:0/3745363067 >> v1:172.21.15.172:6789/0 conn(0x7f30f0069c90 legacy=0x7f30f006d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30eeffd700 1 -- 172.21.15.7:0/3745363067 >> v1:172.21.15.7:6789/0 conn(0x7f30f006b610 legacy=0x7f30f006cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30eeffd700 1 -- 172.21.15.7:0/3745363067 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00c89f0 con 0x7f30f015d3b0 2023-10-07T08:00:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30eeffd700 1 --2- 172.21.15.7:0/3745363067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f015d780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009910 tx=0x7f30e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8aa8bad46c94554 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30edffb700 1 -- 172.21.15.7:0/3745363067 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8001470 con 0x7f30f015d3b0 2023-10-07T08:00:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30edffb700 1 -- 172.21.15.7:0/3745363067 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f30e0011420 con 0x7f30f015d3b0 2023-10-07T08:00:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.596+0000 7f30edffb700 1 -- 172.21.15.7:0/3745363067 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e00117c0 con 0x7f30f015d3b0 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.597+0000 7f30f5968700 1 -- 172.21.15.7:0/3745363067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 msgr2=0x7f30f015d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.597+0000 7f30f5968700 1 --2- 172.21.15.7:0/3745363067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f015d780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009910 tx=0x7f30e00091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.597+0000 7f30f5968700 1 -- 172.21.15.7:0/3745363067 shutdown_connections 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.597+0000 7f30f5968700 1 --2- 172.21.15.7:0/3745363067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f015d780 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.597+0000 7f30f5968700 1 -- 172.21.15.7:0/3745363067 >> 172.21.15.7:0/3745363067 conn(0x7f30f0067ac0 msgr2=0x7f30f00ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.597+0000 7f30f5968700 1 -- 172.21.15.7:0/3745363067 shutdown_connections 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.597+0000 7f30f5968700 1 -- 172.21.15.7:0/3745363067 wait complete. 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.598+0000 7f30f5968700 1 Processor -- start 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.598+0000 7f30f5968700 1 -- start start 2023-10-07T08:00:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.599+0000 7f30f5968700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f0069c90 0x7f30f0151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.599+0000 7f30f5968700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f006b610 0x7f30f00c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.599+0000 7f30f5968700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f016cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.599+0000 7f30f5968700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f00c5080 con 0x7f30f006b610 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.599+0000 7f30f5968700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f00c5ec0 con 0x7f30f0069c90 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.599+0000 7f30f5968700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f0151a60 con 0x7f30f015d3b0 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.599+0000 7f30ee7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f006b610 0x7f30f00c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.599+0000 7f30ee7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f006b610 0x7f30f00c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48160/0 (socket says 172.21.15.7:48160) 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.599+0000 7f30ee7fc700 1 -- 172.21.15.7:0/1478919528 learned_addr learned my addr 172.21.15.7:0/1478919528 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30eeffd700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f0069c90 0x7f30f0151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30ef7fe700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f016cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30ee7fc700 1 -- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 msgr2=0x7f30f016cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30ee7fc700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f016cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30ee7fc700 1 -- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f0069c90 msgr2=0x7f30f0151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30ee7fc700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f0069c90 0x7f30f0151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30ee7fc700 1 -- 172.21.15.7:0/1478919528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e0009420 con 0x7f30f006b610 2023-10-07T08:00:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30ef7fe700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f016cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:00:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30eeffd700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f0069c90 0x7f30f0151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30ee7fc700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f006b610 0x7f30f00c7820 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f30d800cb90 tx=0x7f30d8001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30e6ffd700 1 -- 172.21.15.7:0/1478919528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8001d00 con 0x7f30f006b610 2023-10-07T08:00:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30e6ffd700 1 -- 172.21.15.7:0/1478919528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f30d8007360 con 0x7f30f006b610 2023-10-07T08:00:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f00c52b0 con 0x7f30f006b610 2023-10-07T08:00:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30e6ffd700 1 -- 172.21.15.7:0/1478919528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8004830 con 0x7f30f006b610 2023-10-07T08:00:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.600+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f00c5870 con 0x7f30f006b610 2023-10-07T08:00:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.601+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f00c6070 con 0x7f30f006b610 2023-10-07T08:00:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.602+0000 7f30e6ffd700 1 -- 172.21.15.7:0/1478919528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30d8004be0 con 0x7f30f006b610 2023-10-07T08:00:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.602+0000 7f30e6ffd700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f30d0066f30 0x7f30d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.602+0000 7f30e6ffd700 1 -- 172.21.15.7:0/1478919528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f30d8083a50 con 0x7f30f006b610 2023-10-07T08:00:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.602+0000 7f30eeffd700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f30d0066f30 0x7f30d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.603+0000 7f30eeffd700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f30d0066f30 0x7f30d00693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009860 tx=0x7f30e00091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.607+0000 7f30e6ffd700 1 -- 172.21.15.7:0/1478919528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30f00c6070 con 0x7f30f006b610 2023-10-07T08:00:57.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.808+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_completed_requests"} v 0) v1 -- 0x7f30f00c6030 con 0x7f30f006b610 2023-10-07T08:00:57.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.812+0000 7f30e6ffd700 1 -- 172.21.15.7:0/1478919528 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_completed_requests"}]=0 v20) v1 ==== 109+0+0 (secure 0 0 0) 0x7f30d8018020 con 0x7f30f006b610 2023-10-07T08:00:57.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f30d0066f30 msgr2=0x7f30d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f30d0066f30 0x7f30d00693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009860 tx=0x7f30e00091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f006b610 msgr2=0x7f30f00c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f006b610 0x7f30f00c7820 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f30d800cb90 tx=0x7f30d8001ab0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 shutdown_connections 2023-10-07T08:00:57.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f30d0066f30 0x7f30d00693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30f015d3b0 0x7f30f016cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f0069c90 0x7f30f0151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 --2- 172.21.15.7:0/1478919528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30f006b610 0x7f30f00c7820 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 >> 172.21.15.7:0/1478919528 conn(0x7f30f0067ac0 msgr2=0x7f30f0068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:57.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 shutdown_connections 2023-10-07T08:00:57.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.814+0000 7f30f5968700 1 -- 172.21.15.7:0/1478919528 wait complete. 2023-10-07T08:00:57.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm client client_inject_fixed_oldest_tid 2023-10-07T08:00:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.963+0000 7f280462f700 1 Processor -- start 2023-10-07T08:00:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.963+0000 7f280462f700 1 -- start start 2023-10-07T08:00:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f280462f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc068710 0x7f27fc0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f280462f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27fc0cab00 con 0x7f27fc15c920 2023-10-07T08:00:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f280462f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27fc06a7a0 con 0x7f27fc15ccf0 2023-10-07T08:00:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f280462f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc0cb3c0 con 0x7f27fc068710 2023-10-07T08:00:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f28023cb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f27fc15c920 0x7f27fc068000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44098/0 (socket says 172.21.15.7:44098) 2023-10-07T08:00:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f28023cb700 1 -- 172.21.15.7:0/155888243 learned_addr learned my addr 172.21.15.7:0/155888243 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f2801bca700 1 --2- 172.21.15.7:0/155888243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc068710 0x7f27fc0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f28013c9700 1 -- 172.21.15.7:0/155888243 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492506748 0 0) 0x7f27fc0cab00 con 0x7f27fc15c920 2023-10-07T08:00:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f28013c9700 1 -- 172.21.15.7:0/155888243 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e8003570 con 0x7f27fc15c920 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f28013c9700 1 -- 172.21.15.7:0/155888243 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477080910 0 0) 0x7f27fc06a7a0 con 0x7f27fc15ccf0 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f28013c9700 1 -- 172.21.15.7:0/155888243 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27fc0cab00 con 0x7f27fc15ccf0 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.964+0000 7f2801bca700 1 -- 172.21.15.7:0/155888243 >> v1:172.21.15.172:6789/0 conn(0x7f27fc15ccf0 legacy=0x7f27fc06ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.965+0000 7f2801bca700 1 -- 172.21.15.7:0/155888243 >> v1:172.21.15.7:6789/0 conn(0x7f27fc15c920 legacy=0x7f27fc068000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.965+0000 7f28013c9700 1 -- 172.21.15.7:0/155888243 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3726522794 0 0) 0x7f27e8003570 con 0x7f27fc15c920 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.965+0000 7f2801bca700 1 -- 172.21.15.7:0/155888243 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc0cae00 con 0x7f27fc068710 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.965+0000 7f2801bca700 1 --2- 172.21.15.7:0/155888243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc068710 0x7f27fc0ca520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009910 tx=0x7f27f80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e6ac3756ce40e81 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.965+0000 7f28013c9700 1 -- 172.21.15.7:0/155888243 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0015f0 con 0x7f27fc068710 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.965+0000 7f28013c9700 1 -- 172.21.15.7:0/155888243 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f27f8011420 con 0x7f27fc068710 2023-10-07T08:00:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.965+0000 7f28013c9700 1 -- 172.21.15.7:0/155888243 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80117c0 con 0x7f27fc068710 2023-10-07T08:00:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.966+0000 7f280462f700 1 -- 172.21.15.7:0/155888243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc068710 msgr2=0x7f27fc0ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.966+0000 7f280462f700 1 --2- 172.21.15.7:0/155888243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc068710 0x7f27fc0ca520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009910 tx=0x7f27f80091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.966+0000 7f280462f700 1 -- 172.21.15.7:0/155888243 shutdown_connections 2023-10-07T08:00:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.966+0000 7f280462f700 1 --2- 172.21.15.7:0/155888243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc068710 0x7f27fc0ca520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.966+0000 7f280462f700 1 -- 172.21.15.7:0/155888243 >> 172.21.15.7:0/155888243 conn(0x7f27fc0c9c30 msgr2=0x7f27fc06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.966+0000 7f280462f700 1 -- 172.21.15.7:0/155888243 shutdown_connections 2023-10-07T08:00:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.966+0000 7f280462f700 1 -- 172.21.15.7:0/155888243 wait complete. 2023-10-07T08:00:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.967+0000 7f280462f700 1 Processor -- start 2023-10-07T08:00:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.967+0000 7f280462f700 1 -- start start 2023-10-07T08:00:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f280462f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27fc068710 0x7f27fc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f280462f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc15c920 0x7f27fc0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f280462f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc15ccf0 0x7f27fc0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f280462f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc1593f0 con 0x7f27fc068710 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f280462f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc0b0730 con 0x7f27fc15ccf0 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f280462f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc0afb20 con 0x7f27fc15c920 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f28023cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27fc068710 0x7f27fc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f28023cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27fc068710 0x7f27fc0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48166/0 (socket says 172.21.15.7:48166) 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f28023cb700 1 -- 172.21.15.7:0/3317434248 learned_addr learned my addr 172.21.15.7:0/3317434248 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f2802bcc700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc15ccf0 0x7f27fc0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f2801bca700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc15c920 0x7f27fc0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f28023cb700 1 -- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc15c920 msgr2=0x7f27fc0b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f28023cb700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc15c920 0x7f27fc0b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f28023cb700 1 -- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc15ccf0 msgr2=0x7f27fc0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f28023cb700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc15ccf0 0x7f27fc0af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f28023cb700 1 -- 172.21.15.7:0/3317434248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f8009420 con 0x7f27fc068710 2023-10-07T08:00:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.968+0000 7f28023cb700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27fc068710 0x7f27fc0b0bc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00cb60 tx=0x7f27ec002ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.969+0000 7f27f27fc700 1 -- 172.21.15.7:0/3317434248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0070f0 con 0x7f27fc068710 2023-10-07T08:00:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.969+0000 7f27f27fc700 1 -- 172.21.15.7:0/3317434248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f27ec007250 con 0x7f27fc068710 2023-10-07T08:00:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.969+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27fc0afe00 con 0x7f27fc068710 2023-10-07T08:00:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.969+0000 7f27f27fc700 1 -- 172.21.15.7:0/3317434248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0075f0 con 0x7f27fc068710 2023-10-07T08:00:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.969+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc0b03c0 con 0x7f27fc068710 2023-10-07T08:00:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.970+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27fc04c3c0 con 0x7f27fc068710 2023-10-07T08:00:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.971+0000 7f27f27fc700 1 -- 172.21.15.7:0/3317434248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27ec005a00 con 0x7f27fc068710 2023-10-07T08:00:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.971+0000 7f27f27fc700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f27e0066f30 0x7f27e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.971+0000 7f27f27fc700 1 -- 172.21.15.7:0/3317434248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f27ec084880 con 0x7f27fc068710 2023-10-07T08:00:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.971+0000 7f2801bca700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f27e0066f30 0x7f27e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.972+0000 7f2801bca700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f27e0066f30 0x7f27e00693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009270 tx=0x7f27f80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:57.975+0000 7f27f27fc700 1 -- 172.21.15.7:0/3317434248 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27ec051f50 con 0x7f27fc068710 2023-10-07T08:00:58.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T08:00:58.161 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T08:00:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.208+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "client", "name": "client_inject_fixed_oldest_tid"} v 0) v1 -- 0x7f27fc0b0a10 con 0x7f27fc068710 2023-10-07T08:00:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.211+0000 7f27f27fc700 1 -- 172.21.15.7:0/3317434248 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ec051f50 con 0x7f27fc068710 2023-10-07T08:00:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.212+0000 7f27f27fc700 1 -- 172.21.15.7:0/3317434248 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "client", "name": "client_inject_fixed_oldest_tid"}]=0 v21) v1 ==== 116+0+0 (secure 0 0 0) 0x7f27ec005cb0 con 0x7f27fc068710 2023-10-07T08:00:58.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f27e0066f30 msgr2=0x7f27e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f27e0066f30 0x7f27e00693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009270 tx=0x7f27f80091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27fc068710 msgr2=0x7f27fc0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27fc068710 0x7f27fc0b0bc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00cb60 tx=0x7f27ec002ac0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 shutdown_connections 2023-10-07T08:00:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f27e0066f30 0x7f27e00693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27fc15ccf0 0x7f27fc0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27fc068710 0x7f27fc0b0bc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 --2- 172.21.15.7:0/3317434248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27fc15c920 0x7f27fc0b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 >> 172.21.15.7:0/3317434248 conn(0x7f27fc0c9c30 msgr2=0x7f27fc157600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 shutdown_connections 2023-10-07T08:00:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.214+0000 7f280462f700 1 -- 172.21.15.7:0/3317434248 wait complete. 2023-10-07T08:00:58.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-07T08:00:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.406+0000 7f5185363700 1 Processor -- start 2023-10-07T08:00:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.406+0000 7f5185363700 1 -- start start 2023-10-07T08:00:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.407+0000 7f5185363700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 0x7f51800cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.407+0000 7f5185363700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f51800af810 0x7f51800ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.407+0000 7f5185363700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51800cf1e0 con 0x7f518015dbf0 2023-10-07T08:00:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.407+0000 7f5185363700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800ce5d0 con 0x7f51800af810 2023-10-07T08:00:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.407+0000 7f5185363700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51800ce8b0 con 0x7f51800af440 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.407+0000 7f517e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f518015dbf0 0x7f51800afd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44110/0 (socket says 172.21.15.7:44110) 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.407+0000 7f517e7fc700 1 -- 172.21.15.7:0/3814707700 learned_addr learned my addr 172.21.15.7:0/3814707700 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.407+0000 7f517f7fe700 1 --2- 172.21.15.7:0/3814707700 >> v2:172.21.15.172:3300/0 conn(0x7f51800af810 0x7f51800ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.407+0000 7f517effd700 1 --2- 172.21.15.7:0/3814707700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 0x7f51800cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517dffb700 1 -- 172.21.15.7:0/3814707700 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614085720 0 0) 0x7f51800cf1e0 con 0x7f518015dbf0 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517dffb700 1 -- 172.21.15.7:0/3814707700 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f516c003570 con 0x7f518015dbf0 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517f7fe700 1 -- 172.21.15.7:0/3814707700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 msgr2=0x7f51800cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517f7fe700 1 --2- 172.21.15.7:0/3814707700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 0x7f51800cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517f7fe700 1 -- 172.21.15.7:0/3814707700 >> v1:172.21.15.7:6789/0 conn(0x7f518015dbf0 legacy=0x7f51800afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517f7fe700 1 -- 172.21.15.7:0/3814707700 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800ceb90 con 0x7f51800af810 2023-10-07T08:00:58.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517f7fe700 1 --2- 172.21.15.7:0/3814707700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51800af810 0x7f51800ccc70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5178009e60 tx=0x7f5178009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770c9ecc5eea4ef9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:58.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517dffb700 1 -- 172.21.15.7:0/3814707700 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517800e070 con 0x7f51800af810 2023-10-07T08:00:58.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517dffb700 1 -- 172.21.15.7:0/3814707700 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517800bb40 con 0x7f51800af810 2023-10-07T08:00:58.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.408+0000 7f517dffb700 1 -- 172.21.15.7:0/3814707700 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178012430 con 0x7f51800af810 2023-10-07T08:00:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.409+0000 7f5185363700 1 -- 172.21.15.7:0/3814707700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51800af810 msgr2=0x7f51800ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.409+0000 7f5185363700 1 --2- 172.21.15.7:0/3814707700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51800af810 0x7f51800ccc70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5178009e60 tx=0x7f5178009780 comp rx=0 tx=0).stop 2023-10-07T08:00:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.409+0000 7f5185363700 1 -- 172.21.15.7:0/3814707700 shutdown_connections 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.409+0000 7f5185363700 1 --2- 172.21.15.7:0/3814707700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51800af810 0x7f51800ccc70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.409+0000 7f5185363700 1 --2- 172.21.15.7:0/3814707700 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 0x7f51800cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.409+0000 7f5185363700 1 -- 172.21.15.7:0/3814707700 >> 172.21.15.7:0/3814707700 conn(0x7f5180069220 msgr2=0x7f5180069620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.409+0000 7f5185363700 1 -- 172.21.15.7:0/3814707700 shutdown_connections 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.409+0000 7f5185363700 1 -- 172.21.15.7:0/3814707700 wait complete. 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.410+0000 7f5185363700 1 Processor -- start 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.410+0000 7f5185363700 1 -- start start 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.411+0000 7f5185363700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 0x7f51800b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.411+0000 7f5185363700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51800af810 0x7f5180147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.411+0000 7f5185363700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015dbf0 0x7f5180157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.411+0000 7f5185363700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800b59b0 con 0x7f51800af810 2023-10-07T08:00:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.411+0000 7f5185363700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800b4fe0 con 0x7f518015dbf0 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.411+0000 7f5185363700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51800b3f10 con 0x7f51800af440 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.411+0000 7f517e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51800af810 0x7f5180147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.411+0000 7f517e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51800af810 0x7f5180147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48172/0 (socket says 172.21.15.7:48172) 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.411+0000 7f517e7fc700 1 -- 172.21.15.7:0/2539014868 learned_addr learned my addr 172.21.15.7:0/2539014868 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f517f7fe700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015dbf0 0x7f5180157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f517effd700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 0x7f51800b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f517e7fc700 1 -- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 msgr2=0x7f51800b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f517e7fc700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 0x7f51800b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f517e7fc700 1 -- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015dbf0 msgr2=0x7f5180157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f517e7fc700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015dbf0 0x7f5180157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f517e7fc700 1 -- 172.21.15.7:0/2539014868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5178009a20 con 0x7f51800af810 2023-10-07T08:00:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f517e7fc700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51800af810 0x7f5180147cc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f517000c7c0 tx=0x7f5170007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f5176ffd700 1 -- 172.21.15.7:0/2539014868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170005b30 con 0x7f51800af810 2023-10-07T08:00:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f5176ffd700 1 -- 172.21.15.7:0/2539014868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5170005c90 con 0x7f51800af810 2023-10-07T08:00:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51800b4990 con 0x7f51800af810 2023-10-07T08:00:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f5176ffd700 1 -- 172.21.15.7:0/2539014868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170007340 con 0x7f51800af810 2023-10-07T08:00:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.412+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5180148200 con 0x7f51800af810 2023-10-07T08:00:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.413+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51800b5170 con 0x7f51800af810 2023-10-07T08:00:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.414+0000 7f5176ffd700 1 -- 172.21.15.7:0/2539014868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f517001b400 con 0x7f51800af810 2023-10-07T08:00:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.414+0000 7f5176ffd700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5160066f30 0x7f51600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.414+0000 7f5176ffd700 1 -- 172.21.15.7:0/2539014868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5170083d40 con 0x7f51800af810 2023-10-07T08:00:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.415+0000 7f517effd700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5160066f30 0x7f51600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.415+0000 7f517effd700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5160066f30 0x7f51600693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5168009ca0 tx=0x7f5168008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.419+0000 7f5176ffd700 1 -- 172.21.15.7:0/2539014868 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5170051330 con 0x7f51800af810 2023-10-07T08:00:58.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.643+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f518004c3c0 con 0x7f51800af810 2023-10-07T08:00:58.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.648+0000 7f5176ffd700 1 -- 172.21.15.7:0/2539014868 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5170054950 con 0x7f51800af810 2023-10-07T08:00:58.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5160066f30 msgr2=0x7f51600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5160066f30 0x7f51600693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5168009ca0 tx=0x7f5168008040 comp rx=0 tx=0).stop 2023-10-07T08:00:58.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51800af810 msgr2=0x7f5180147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51800af810 0x7f5180147cc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f517000c7c0 tx=0x7f5170007e00 comp rx=0 tx=0).stop 2023-10-07T08:00:58.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 shutdown_connections 2023-10-07T08:00:58.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f5160066f30 0x7f51600693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015dbf0 0x7f5180157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51800af440 0x7f51800b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 --2- 172.21.15.7:0/2539014868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51800af810 0x7f5180147cc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.650+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 >> 172.21.15.7:0/2539014868 conn(0x7f5180069220 msgr2=0x7f51800c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:58.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.651+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 shutdown_connections 2023-10-07T08:00:58.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.651+0000 7f5185363700 1 -- 172.21.15.7:0/2539014868 wait complete. 2023-10-07T08:00:58.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_metadata_threshold 2023-10-07T08:00:58.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.799+0000 7f71971a1700 1 Processor -- start 2023-10-07T08:00:58.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.799+0000 7f71971a1700 1 -- start start 2023-10-07T08:00:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f71971a1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7190156df0 0x7f71900b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f71971a1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f71901571c0 0x7f71900cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f71971a1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71900ce3d0 0x7f71900b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f71971a1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71900b3b90 con 0x7f71900ce3d0 2023-10-07T08:00:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f71971a1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71900ce850 con 0x7f71901571c0 2023-10-07T08:00:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f71971a1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71900b1790 con 0x7f7190156df0 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f71969a0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71900ce3d0 0x7f71900b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f71969a0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71900ce3d0 0x7f71900b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48176/0 (socket says 172.21.15.7:48176) 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f71969a0700 1 -- 172.21.15.7:0/2834372253 learned_addr learned my addr 172.21.15.7:0/2834372253 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f719619f700 1 --2- 172.21.15.7:0/2834372253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7190156df0 0x7f71900b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.800+0000 7f719599e700 1 --2- 172.21.15.7:0/2834372253 >> v2:172.21.15.172:3300/0 conn(0x7f71901571c0 0x7f71900cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.801+0000 7f71969a0700 1 -- 172.21.15.7:0/2834372253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7190156df0 msgr2=0x7f71900b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.801+0000 7f71969a0700 1 --2- 172.21.15.7:0/2834372253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7190156df0 0x7f71900b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.801+0000 7f71969a0700 1 -- 172.21.15.7:0/2834372253 >> v2:172.21.15.172:3300/0 conn(0x7f71901571c0 msgr2=0x7f71900cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.801+0000 7f71969a0700 1 --2- 172.21.15.7:0/2834372253 >> v2:172.21.15.172:3300/0 conn(0x7f71901571c0 0x7f71900cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.801+0000 7f71969a0700 1 -- 172.21.15.7:0/2834372253 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71900b0e60 con 0x7f71900ce3d0 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.801+0000 7f71969a0700 1 --2- 172.21.15.7:0/2834372253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 0x7f71900b2f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f719006a3a0 tx=0x7f7188009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a63efab64a6367 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.801+0000 7f719519d700 1 -- 172.21.15.7:0/2834372253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718800e070 con 0x7f71900ce3d0 2023-10-07T08:00:58.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.801+0000 7f719519d700 1 -- 172.21.15.7:0/2834372253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718800b930 con 0x7f71900ce3d0 2023-10-07T08:00:58.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.801+0000 7f719519d700 1 -- 172.21.15.7:0/2834372253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71880092e0 con 0x7f71900ce3d0 2023-10-07T08:00:58.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.802+0000 7f71971a1700 1 -- 172.21.15.7:0/2834372253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 msgr2=0x7f71900b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.802+0000 7f71971a1700 1 --2- 172.21.15.7:0/2834372253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 0x7f71900b2f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f719006a3a0 tx=0x7f7188009840 comp rx=0 tx=0).stop 2023-10-07T08:00:58.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.802+0000 7f71971a1700 1 -- 172.21.15.7:0/2834372253 shutdown_connections 2023-10-07T08:00:58.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.802+0000 7f71971a1700 1 --2- 172.21.15.7:0/2834372253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 0x7f71900b2f40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.802+0000 7f71971a1700 1 --2- 172.21.15.7:0/2834372253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7190156df0 0x7f71900b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.802+0000 7f71971a1700 1 --2- 172.21.15.7:0/2834372253 >> v2:172.21.15.172:3300/0 conn(0x7f71901571c0 0x7f71900cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.802+0000 7f71971a1700 1 -- 172.21.15.7:0/2834372253 >> 172.21.15.7:0/2834372253 conn(0x7f7190066ff0 msgr2=0x7f71900673f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.802+0000 7f71971a1700 1 -- 172.21.15.7:0/2834372253 shutdown_connections 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.802+0000 7f71971a1700 1 -- 172.21.15.7:0/2834372253 wait complete. 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.803+0000 7f71971a1700 1 Processor -- start 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.803+0000 7f71971a1700 1 -- start start 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f71971a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 0x7f71900c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f71971a1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7190156df0 0x7f719016a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f71971a1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71901571c0 0x7f71901470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f71971a1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719016ace0 con 0x7f71900ce3d0 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f71971a1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71900c7150 con 0x7f7190156df0 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f71971a1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71900c7430 con 0x7f71901571c0 2023-10-07T08:00:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719619f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 0x7f71900c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719619f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 0x7f71900c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48178/0 (socket says 172.21.15.7:48178) 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719619f700 1 -- 172.21.15.7:0/3062968695 learned_addr learned my addr 172.21.15.7:0/3062968695 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719599e700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7190156df0 0x7f719016a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f71969a0700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71901571c0 0x7f71901470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719619f700 1 -- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71901571c0 msgr2=0x7f71901470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719619f700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71901571c0 0x7f71901470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719619f700 1 -- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7190156df0 msgr2=0x7f719016a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719619f700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7190156df0 0x7f719016a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719619f700 1 -- 172.21.15.7:0/3062968695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188009ae0 con 0x7f71900ce3d0 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.804+0000 7f719599e700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7190156df0 0x7f719016a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.805+0000 7f719619f700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 0x7f71900c98c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f718c00ec80 tx=0x7f718c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.805+0000 7f71867fc700 1 -- 172.21.15.7:0/3062968695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c00c9d0 con 0x7f71900ce3d0 2023-10-07T08:00:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.805+0000 7f71867fc700 1 -- 172.21.15.7:0/3062968695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c00cb30 con 0x7f71900ce3d0 2023-10-07T08:00:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.805+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71900c7710 con 0x7f71900ce3d0 2023-10-07T08:00:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.805+0000 7f71867fc700 1 -- 172.21.15.7:0/3062968695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c013470 con 0x7f71900ce3d0 2023-10-07T08:00:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.805+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71900c7aa0 con 0x7f71900ce3d0 2023-10-07T08:00:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.806+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71900c7e90 con 0x7f71900ce3d0 2023-10-07T08:00:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.806+0000 7f71867fc700 1 -- 172.21.15.7:0/3062968695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f718c013820 con 0x7f71900ce3d0 2023-10-07T08:00:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.807+0000 7f71867fc700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7174066f30 0x7f71740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.807+0000 7f71867fc700 1 -- 172.21.15.7:0/3062968695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f718c0845c0 con 0x7f71900ce3d0 2023-10-07T08:00:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.807+0000 7f719599e700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7174066f30 0x7f71740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.808+0000 7f719599e700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7174066f30 0x7f71740693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f71901568a0 tx=0x7f7180008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:58.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:58.811+0000 7f71867fc700 1 -- 172.21.15.7:0/3062968695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f718c051b30 con 0x7f71900ce3d0 2023-10-07T08:00:59.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.057+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_metadata_threshold"} v 0) v1 -- 0x7f719004c3c0 con 0x7f71900ce3d0 2023-10-07T08:00:59.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.061+0000 7f71867fc700 1 -- 172.21.15.7:0/3062968695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_metadata_threshold"}]=0 v23) v1 ==== 113+0+0 (secure 0 0 0) 0x7f718c055150 con 0x7f71900ce3d0 2023-10-07T08:00:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.063+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7174066f30 msgr2=0x7f71740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7174066f30 0x7f71740693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f71901568a0 tx=0x7f7180008040 comp rx=0 tx=0).stop 2023-10-07T08:00:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 msgr2=0x7f71900c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 0x7f71900c98c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f718c00ec80 tx=0x7f718c0091a0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 shutdown_connections 2023-10-07T08:00:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7174066f30 0x7f71740693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71901571c0 0x7f71901470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900ce3d0 0x7f71900c98c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 --2- 172.21.15.7:0/3062968695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7190156df0 0x7f719016a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 >> 172.21.15.7:0/3062968695 conn(0x7f7190066ff0 msgr2=0x7f7190157700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 shutdown_connections 2023-10-07T08:00:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.064+0000 7f71971a1700 1 -- 172.21.15.7:0/3062968695 wait complete. 2023-10-07T08:00:59.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2023-10-07T08:00:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.210+0000 7f4c653de700 1 Processor -- start 2023-10-07T08:00:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.211+0000 7f4c653de700 1 -- start start 2023-10-07T08:00:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.211+0000 7f4c653de700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 0x7f4c600b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.211+0000 7f4c653de700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c600d0250 0x7f4c600d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.211+0000 7f4c653de700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4c600b3b60 0x7f4c600b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.211+0000 7f4c653de700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c600d0be0 con 0x7f4c600d0250 2023-10-07T08:00:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.211+0000 7f4c653de700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c600b4820 con 0x7f4c600b3b60 2023-10-07T08:00:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.211+0000 7f4c653de700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600b5000 con 0x7f4c60159380 2023-10-07T08:00:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c600d0250 0x7f4c600d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c600d0250 0x7f4c600d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48184/0 (socket says 172.21.15.7:48184) 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5e7fc700 1 -- 172.21.15.7:0/1544905783 learned_addr learned my addr 172.21.15.7:0/1544905783 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5f7fe700 1 --2- 172.21.15.7:0/1544905783 >> v2:172.21.15.172:3300/0 conn(0x7f4c600b3b60 0x7f4c600b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5effd700 1 --2- 172.21.15.7:0/1544905783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 0x7f4c600b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5e7fc700 1 -- 172.21.15.7:0/1544905783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 msgr2=0x7f4c600b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5e7fc700 1 --2- 172.21.15.7:0/1544905783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 0x7f4c600b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5e7fc700 1 -- 172.21.15.7:0/1544905783 >> v2:172.21.15.172:3300/0 conn(0x7f4c600b3b60 msgr2=0x7f4c600b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5e7fc700 1 --2- 172.21.15.7:0/1544905783 >> v2:172.21.15.172:3300/0 conn(0x7f4c600b3b60 0x7f4c600b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5e7fc700 1 -- 172.21.15.7:0/1544905783 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600b52e0 con 0x7f4c600d0250 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5f7fe700 1 --2- 172.21.15.7:0/1544905783 >> v2:172.21.15.172:3300/0 conn(0x7f4c600b3b60 0x7f4c600b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5e7fc700 1 --2- 172.21.15.7:0/1544905783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 0x7f4c600d06a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4c600b34a0 tx=0x7f4c50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9494d36a9b37003c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5dffb700 1 -- 172.21.15.7:0/1544905783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000b860 con 0x7f4c600d0250 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5dffb700 1 -- 172.21.15.7:0/1544905783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5000b9c0 con 0x7f4c600d0250 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.212+0000 7f4c5dffb700 1 -- 172.21.15.7:0/1544905783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000bd60 con 0x7f4c600d0250 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.213+0000 7f4c653de700 1 -- 172.21.15.7:0/1544905783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 msgr2=0x7f4c600d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.213+0000 7f4c653de700 1 --2- 172.21.15.7:0/1544905783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 0x7f4c600d06a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4c600b34a0 tx=0x7f4c50009250 comp rx=0 tx=0).stop 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.213+0000 7f4c653de700 1 -- 172.21.15.7:0/1544905783 shutdown_connections 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.213+0000 7f4c653de700 1 --2- 172.21.15.7:0/1544905783 >> v2:172.21.15.172:3300/0 conn(0x7f4c600b3b60 0x7f4c600b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.213+0000 7f4c653de700 1 --2- 172.21.15.7:0/1544905783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 0x7f4c600b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.213+0000 7f4c653de700 1 --2- 172.21.15.7:0/1544905783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 0x7f4c600d06a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.213+0000 7f4c653de700 1 -- 172.21.15.7:0/1544905783 >> 172.21.15.7:0/1544905783 conn(0x7f4c60067be0 msgr2=0x7f4c60067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.213+0000 7f4c653de700 1 -- 172.21.15.7:0/1544905783 shutdown_connections 2023-10-07T08:00:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.213+0000 7f4c653de700 1 -- 172.21.15.7:0/1544905783 wait complete. 2023-10-07T08:00:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.214+0000 7f4c653de700 1 Processor -- start 2023-10-07T08:00:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.214+0000 7f4c653de700 1 -- start start 2023-10-07T08:00:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.215+0000 7f4c653de700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c600b3b60 0x7f4c60152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.215+0000 7f4c653de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 0x7f4c601580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.215+0000 7f4c653de700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 0x7f4c601585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.215+0000 7f4c653de700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6015c1f0 con 0x7f4c600d0250 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.215+0000 7f4c653de700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60156120 con 0x7f4c600b3b60 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.215+0000 7f4c653de700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c601557f0 con 0x7f4c60159380 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.215+0000 7f4c5e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 0x7f4c601580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.215+0000 7f4c5e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 0x7f4c601580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48188/0 (socket says 172.21.15.7:48188) 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.215+0000 7f4c5e7fc700 1 -- 172.21.15.7:0/2987263905 learned_addr learned my addr 172.21.15.7:0/2987263905 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c5effd700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c600b3b60 0x7f4c60152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c5f7fe700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 0x7f4c601585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c5e7fc700 1 -- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 msgr2=0x7f4c601585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c5e7fc700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 0x7f4c601585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c5e7fc700 1 -- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c600b3b60 msgr2=0x7f4c60152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:00:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c5e7fc700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c600b3b60 0x7f4c60152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c5e7fc700 1 -- 172.21.15.7:0/2987263905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c500094f0 con 0x7f4c600d0250 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c5effd700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c600b3b60 0x7f4c60152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c5e7fc700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 0x7f4c601580b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009290 tx=0x7f4c50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c46ffd700 1 -- 172.21.15.7:0/2987263905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50026480 con 0x7f4c600d0250 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c46ffd700 1 -- 172.21.15.7:0/2987263905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5001f070 con 0x7f4c600d0250 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c46ffd700 1 -- 172.21.15.7:0/2987263905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50026830 con 0x7f4c600d0250 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c60155ad0 con 0x7f4c600d0250 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.216+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6016a9b0 con 0x7f4c600d0250 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.217+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6016ac60 con 0x7f4c600d0250 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.218+0000 7f4c46ffd700 1 -- 172.21.15.7:0/2987263905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c50026ba0 con 0x7f4c600d0250 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.218+0000 7f4c46ffd700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c4c066f30 0x7f4c4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:00:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.218+0000 7f4c46ffd700 1 -- 172.21.15.7:0/2987263905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4c50021070 con 0x7f4c600d0250 2023-10-07T08:00:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.218+0000 7f4c5effd700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c4c066f30 0x7f4c4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:00:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.219+0000 7f4c5effd700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c4c066f30 0x7f4c4c0693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4c54006fd0 tx=0x7f4c54008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:00:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.222+0000 7f4c46ffd700 1 -- 172.21.15.7:0/2987263905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c6016ac60 con 0x7f4c600d0250 2023-10-07T08:00:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:00:59.510+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) v1 -- 0x7f4c6016ac60 con 0x7f4c600d0250 2023-10-07T08:01:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.442+0000 7f4c46ffd700 1 -- 172.21.15.7:0/2987263905 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v159) v1 ==== 162+0+0 (secure 0 0 0) 0x7f4c6016ac60 con 0x7f4c600d0250 2023-10-07T08:01:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c4c066f30 msgr2=0x7f4c4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c4c066f30 0x7f4c4c0693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4c54006fd0 tx=0x7f4c54008040 comp rx=0 tx=0).stop 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 msgr2=0x7f4c601580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 0x7f4c601580b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009290 tx=0x7f4c50017000 comp rx=0 tx=0).stop 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 shutdown_connections 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4c4c066f30 0x7f4c4c0693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c60159380 0x7f4c601585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c600b3b60 0x7f4c60152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 --2- 172.21.15.7:0/2987263905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c600d0250 0x7f4c601580b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 >> 172.21.15.7:0/2987263905 conn(0x7f4c60067be0 msgr2=0x7f4c600b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 shutdown_connections 2023-10-07T08:01:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:00.445+0000 7f4c653de700 1 -- 172.21.15.7:0/2987263905 wait complete. 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the ... FAIL 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner: 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner:FAIL: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks/cephfs/test_client_limits.py", line 280, in test_client_blocklisted_oldest_tid 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner: self.mount_a.create_n_files("testdir/file2", 20, True) 2023-10-07T08:01:00.463 INFO:tasks.cephfs_test_runner:AssertionError: CommandFailedError not raised 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner: 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner:Ran 2 tests in 158.405s 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner: 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner: 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner:FAIL: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-07T08:01:00.464 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks/cephfs/test_client_limits.py", line 280, in test_client_blocklisted_oldest_tid 2023-10-07T08:01:00.465 INFO:tasks.cephfs_test_runner: self.mount_a.create_n_files("testdir/file2", 20, True) 2023-10-07T08:01:00.465 INFO:tasks.cephfs_test_runner:AssertionError: CommandFailedError not raised 2023-10-07T08:01:00.465 INFO:tasks.cephfs_test_runner: 2023-10-07T08:01:00.465 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T08:01:00.532 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=605d114cac0b40c484d7c055708d070a Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T08:01:00.534 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-07T08:01:00.543 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-07T08:01:00.551 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-07T08:01:00.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:01:00.552 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-10-07T08:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-07T08:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T08:01:00.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:01:00.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T08:01:00.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:01:00.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T08:01:00.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:01:00.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2023-10-07T08:01:00.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:01:00.824 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-10-07T08:01:00.847 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-07T08:01:00.847 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-07T08:01:00.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:01:00.848 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-07T08:01:00.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:01:00.931 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T08:01:01.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T08:01:01.016 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2023-10-07T08:01:01.094 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-07T08:01:01.108 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-07T08:01:01.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T08:01:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.251+0000 7f72da277700 1 Processor -- start 2023-10-07T08:01:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.251+0000 7f72da277700 1 -- start start 2023-10-07T08:01:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72da277700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72d4157470 0x7f72d406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72da277700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d406bb60 0x7f72d40c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72da277700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d40c8f10 con 0x7f72d41570a0 2023-10-07T08:01:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72da277700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d406c000 con 0x7f72d4157470 2023-10-07T08:01:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72da277700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40c3b80 con 0x7f72d406bb60 2023-10-07T08:01:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72d3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f72d41570a0 0x7f72d40c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46990/0 (socket says 172.21.15.7:46990) 2023-10-07T08:01:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72d37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72d4157470 0x7f72d406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72d3fff700 1 -- 172.21.15.7:0/3716710975 learned_addr learned my addr 172.21.15.7:0/3716710975 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72d8814700 1 --2- 172.21.15.7:0/3716710975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d406bb60 0x7f72d40c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72d37fe700 1 -- 172.21.15.7:0/3716710975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d406bb60 msgr2=0x7f72d40c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72d37fe700 1 --2- 172.21.15.7:0/3716710975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d406bb60 0x7f72d40c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72d37fe700 1 -- 172.21.15.7:0/3716710975 >> v1:172.21.15.7:6789/0 conn(0x7f72d41570a0 legacy=0x7f72d40c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72d37fe700 1 -- 172.21.15.7:0/3716710975 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40c2f70 con 0x7f72d4157470 2023-10-07T08:01:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.252+0000 7f72d2ffd700 1 -- 172.21.15.7:0/3716710975 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191060438 0 0) 0x7f72c4001110 con 0x7f72d41570a0 2023-10-07T08:01:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72d37fe700 1 --2- 172.21.15.7:0/3716710975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d4157470 0x7f72d406b620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0098e0 tx=0x7f72bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8679b73d27f5841d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72d2ffd700 1 -- 172.21.15.7:0/3716710975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc009e80 con 0x7f72d4157470 2023-10-07T08:01:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72d2ffd700 1 -- 172.21.15.7:0/3716710975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc00b860 con 0x7f72d4157470 2023-10-07T08:01:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72d2ffd700 1 -- 172.21.15.7:0/3716710975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc00bbe0 con 0x7f72d4157470 2023-10-07T08:01:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72da277700 1 -- 172.21.15.7:0/3716710975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d4157470 msgr2=0x7f72d406b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72da277700 1 --2- 172.21.15.7:0/3716710975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d4157470 0x7f72d406b620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0098e0 tx=0x7f72bc0091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72da277700 1 -- 172.21.15.7:0/3716710975 shutdown_connections 2023-10-07T08:01:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72da277700 1 --2- 172.21.15.7:0/3716710975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d406bb60 0x7f72d40c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.253+0000 7f72da277700 1 --2- 172.21.15.7:0/3716710975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d4157470 0x7f72d406b620 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.254+0000 7f72da277700 1 -- 172.21.15.7:0/3716710975 >> 172.21.15.7:0/3716710975 conn(0x7f72d4067ac0 msgr2=0x7f72d40ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.254+0000 7f72da277700 1 -- 172.21.15.7:0/3716710975 shutdown_connections 2023-10-07T08:01:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.254+0000 7f72da277700 1 -- 172.21.15.7:0/3716710975 wait complete. 2023-10-07T08:01:01.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.255+0000 7f72da277700 1 Processor -- start 2023-10-07T08:01:01.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.255+0000 7f72da277700 1 -- start start 2023-10-07T08:01:01.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72da277700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d406bb60 0x7f72d4068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72da277700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d41570a0 0x7f72d4139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72da277700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4157470 0x7f72d415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72da277700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d415a230 con 0x7f72d406bb60 2023-10-07T08:01:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72da277700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40b3df0 con 0x7f72d4157470 2023-10-07T08:01:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72da277700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40b3b10 con 0x7f72d41570a0 2023-10-07T08:01:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d406bb60 0x7f72d4068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d406bb60 0x7f72d4068100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45018/0 (socket says 172.21.15.7:45018) 2023-10-07T08:01:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d3fff700 1 -- 172.21.15.7:0/2701340490 learned_addr learned my addr 172.21.15.7:0/2701340490 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d37fe700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d41570a0 0x7f72d4139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d8814700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4157470 0x7f72d415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d3fff700 1 -- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d41570a0 msgr2=0x7f72d4139240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d3fff700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d41570a0 0x7f72d4139240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d3fff700 1 -- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4157470 msgr2=0x7f72d415a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d3fff700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4157470 0x7f72d415a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d3fff700 1 -- 172.21.15.7:0/2701340490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc009420 con 0x7f72d406bb60 2023-10-07T08:01:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.256+0000 7f72d3fff700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d406bb60 0x7f72d4068100 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c810 tx=0x7f72c4007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.257+0000 7f72cbfff700 1 -- 172.21.15.7:0/2701340490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4005b30 con 0x7f72d406bb60 2023-10-07T08:01:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.257+0000 7f72cbfff700 1 -- 172.21.15.7:0/2701340490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4005cb0 con 0x7f72d406bb60 2023-10-07T08:01:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.257+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d4159b70 con 0x7f72d406bb60 2023-10-07T08:01:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.257+0000 7f72cbfff700 1 -- 172.21.15.7:0/2701340490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4007320 con 0x7f72d406bb60 2023-10-07T08:01:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.257+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d4139780 con 0x7f72d406bb60 2023-10-07T08:01:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.258+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d4139970 con 0x7f72d406bb60 2023-10-07T08:01:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.258+0000 7f72cbfff700 1 -- 172.21.15.7:0/2701340490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72c401b3b0 con 0x7f72d406bb60 2023-10-07T08:01:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.258+0000 7f72cbfff700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f72b4066f30 0x7f72b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.259+0000 7f72cbfff700 1 -- 172.21.15.7:0/2701340490 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f72c40834c0 con 0x7f72d406bb60 2023-10-07T08:01:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.259+0000 7f72d37fe700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f72b4066f30 0x7f72b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.259+0000 7f72d37fe700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f72b4066f30 0x7f72b40693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0091e0 tx=0x7f72bc017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.263+0000 7f72cbfff700 1 -- 172.21.15.7:0/2701340490 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72c4050a30 con 0x7f72d406bb60 2023-10-07T08:01:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.466+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72d40c8f10 con 0x7f72d406bb60 2023-10-07T08:01:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.467+0000 7f72cbfff700 1 -- 172.21.15.7:0/2701340490 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12307 (secure 0 0 0) 0x7f72d40c8f10 con 0x7f72d406bb60 2023-10-07T08:01:01.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:01:01.469 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T08:00:57.430557+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/763397529":"2023-10-08T08:00:54.423343+0000","172.21.15.7:6834/763397529":"2023-10-08T08:00:54.423343+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T08:01:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f72b4066f30 msgr2=0x7f72b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f72b4066f30 0x7f72b40693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0091e0 tx=0x7f72bc017000 comp rx=0 tx=0).stop 2023-10-07T08:01:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d406bb60 msgr2=0x7f72d4068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d406bb60 0x7f72d4068100 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c810 tx=0x7f72c4007c50 comp rx=0 tx=0).stop 2023-10-07T08:01:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 shutdown_connections 2023-10-07T08:01:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f72b4066f30 0x7f72b40693e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4157470 0x7f72d415a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72d406bb60 0x7f72d4068100 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 --2- 172.21.15.7:0/2701340490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72d41570a0 0x7f72d4139240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 >> 172.21.15.7:0/2701340490 conn(0x7f72d4067ac0 msgr2=0x7f72d413d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 shutdown_connections 2023-10-07T08:01:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.469+0000 7f72da277700 1 -- 172.21.15.7:0/2701340490 wait complete. 2023-10-07T08:01:01.484 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-07T08:01:01.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T08:01:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.616+0000 7f19a2498700 1 Processor -- start 2023-10-07T08:01:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.616+0000 7f19a2498700 1 -- start start 2023-10-07T08:01:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.617+0000 7f19a2498700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f199c159b30 0x7f199c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.617+0000 7f19a2498700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 0x7f199c069490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.617+0000 7f19a2498700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c06ea20 con 0x7f199c1533a0 2023-10-07T08:01:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.617+0000 7f19a2498700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199c0c6880 con 0x7f199c159b30 2023-10-07T08:01:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.617+0000 7f19a2498700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c06dbb0 con 0x7f199c06d730 2023-10-07T08:01:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f19a0a35700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 0x7f199c069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f19a0a35700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 0x7f199c069490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:36794/0 (socket says 172.21.15.7:36794) 2023-10-07T08:01:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f19a0a35700 1 -- 172.21.15.7:0/2663803208 learned_addr learned my addr 172.21.15.7:0/2663803208 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f199b7fe700 1 --2- 172.21.15.7:0/2663803208 >> v2:172.21.15.7:3300/0 conn(0x7f199c159b30 0x7f199c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f199b7fe700 1 -- 172.21.15.7:0/2663803208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 msgr2=0x7f199c069490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f199b7fe700 1 --2- 172.21.15.7:0/2663803208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 0x7f199c069490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f199b7fe700 1 -- 172.21.15.7:0/2663803208 >> v1:172.21.15.7:6789/0 conn(0x7f199c1533a0 legacy=0x7f199c159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f199b7fe700 1 -- 172.21.15.7:0/2663803208 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c06de10 con 0x7f199c159b30 2023-10-07T08:01:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f199affd700 1 -- 172.21.15.7:0/2663803208 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077650298 0 0) 0x7f199c06ea20 con 0x7f199c1533a0 2023-10-07T08:01:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f199b7fe700 1 --2- 172.21.15.7:0/2663803208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 0x7f199c06f2e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f19900098e0 tx=0x7f19900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9254087d4cdae8a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.618+0000 7f199affd700 1 -- 172.21.15.7:0/2663803208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199000b640 con 0x7f199c159b30 2023-10-07T08:01:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f199affd700 1 -- 172.21.15.7:0/2663803208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199000b7a0 con 0x7f199c159b30 2023-10-07T08:01:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f199affd700 1 -- 172.21.15.7:0/2663803208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199000bb40 con 0x7f199c159b30 2023-10-07T08:01:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f19a2498700 1 -- 172.21.15.7:0/2663803208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 msgr2=0x7f199c06f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f19a2498700 1 --2- 172.21.15.7:0/2663803208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 0x7f199c06f2e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f19900098e0 tx=0x7f19900091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f19a2498700 1 -- 172.21.15.7:0/2663803208 shutdown_connections 2023-10-07T08:01:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f19a2498700 1 --2- 172.21.15.7:0/2663803208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 0x7f199c069490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f19a2498700 1 --2- 172.21.15.7:0/2663803208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 0x7f199c06f2e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f19a2498700 1 -- 172.21.15.7:0/2663803208 >> 172.21.15.7:0/2663803208 conn(0x7f199c067a80 msgr2=0x7f199c0cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f19a2498700 1 -- 172.21.15.7:0/2663803208 shutdown_connections 2023-10-07T08:01:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.619+0000 7f19a2498700 1 -- 172.21.15.7:0/2663803208 wait complete. 2023-10-07T08:01:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.620+0000 7f19a2498700 1 Processor -- start 2023-10-07T08:01:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.621+0000 7f19a2498700 1 -- start start 2023-10-07T08:01:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.621+0000 7f19a2498700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 0x7f199c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.621+0000 7f19a2498700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c1533a0 0x7f199c0ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.621+0000 7f19a2498700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 0x7f199c0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.621+0000 7f19a2498700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c153ba0 con 0x7f199c159b30 2023-10-07T08:01:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a2498700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c153e40 con 0x7f199c1533a0 2023-10-07T08:01:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a2498700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c154120 con 0x7f199c06d730 2023-10-07T08:01:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a0a35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 0x7f199c0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a0a35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 0x7f199c0abc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45032/0 (socket says 172.21.15.7:45032) 2023-10-07T08:01:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a0a35700 1 -- 172.21.15.7:0/2052491550 learned_addr learned my addr 172.21.15.7:0/2052491550 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f199bfff700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 0x7f199c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f199b7fe700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c1533a0 0x7f199c0ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a0a35700 1 -- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 msgr2=0x7f199c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a0a35700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 0x7f199c1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a0a35700 1 -- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c1533a0 msgr2=0x7f199c0ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a0a35700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c1533a0 0x7f199c0ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a0a35700 1 -- 172.21.15.7:0/2052491550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1990009420 con 0x7f199c159b30 2023-10-07T08:01:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a0a35700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 0x7f199c0abc30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1994009e50 tx=0x7f199400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f1983fff700 1 -- 172.21.15.7:0/2052491550 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199400ca10 con 0x7f199c159b30 2023-10-07T08:01:01.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f1983fff700 1 -- 172.21.15.7:0/2052491550 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199400f040 con 0x7f199c159b30 2023-10-07T08:01:01.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c154400 con 0x7f199c159b30 2023-10-07T08:01:01.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.622+0000 7f1983fff700 1 -- 172.21.15.7:0/2052491550 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199400cdc0 con 0x7f199c159b30 2023-10-07T08:01:01.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.623+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c158b10 con 0x7f199c159b30 2023-10-07T08:01:01.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.624+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199c0cb840 con 0x7f199c159b30 2023-10-07T08:01:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.624+0000 7f1983fff700 1 -- 172.21.15.7:0/2052491550 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1994014650 con 0x7f199c159b30 2023-10-07T08:01:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.624+0000 7f1983fff700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1988066f30 0x7f19880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.625+0000 7f1983fff700 1 -- 172.21.15.7:0/2052491550 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f19940849a0 con 0x7f199c159b30 2023-10-07T08:01:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.625+0000 7f199bfff700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1988066f30 0x7f19880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.625+0000 7f199bfff700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1988066f30 0x7f19880693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f199c0aa980 tx=0x7f198c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.629+0000 7f1983fff700 1 -- 172.21.15.7:0/2052491550 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1994051f10 con 0x7f199c159b30 2023-10-07T08:01:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.828+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f199c0c69c0 con 0x7f1988066f30 2023-10-07T08:01:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.831+0000 7f1983fff700 1 -- 172.21.15.7:0/2052491550 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50762 (secure 0 0 0) 0x7f199c0c69c0 con 0x7f1988066f30 2023-10-07T08:01:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:01:01.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1988066f30 msgr2=0x7f19880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1988066f30 0x7f19880693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f199c0aa980 tx=0x7f198c009040 comp rx=0 tx=0).stop 2023-10-07T08:01:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 msgr2=0x7f199c0abc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 0x7f199c0abc30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1994009e50 tx=0x7f199400b040 comp rx=0 tx=0).stop 2023-10-07T08:01:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 shutdown_connections 2023-10-07T08:01:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1988066f30 0x7f19880693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f199c159b30 0x7f199c0abc30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f199c06d730 0x7f199c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 --2- 172.21.15.7:0/2052491550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c1533a0 0x7f199c0ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 >> 172.21.15.7:0/2052491550 conn(0x7f199c067a80 msgr2=0x7f199c0c20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 shutdown_connections 2023-10-07T08:01:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.833+0000 7f19a2498700 1 -- 172.21.15.7:0/2052491550 wait complete. 2023-10-07T08:01:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T08:01:01.848 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2023-10-07T08:01:00.069604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2529328,"kb_used_data":68144,"kb_used_omap":0,"kb_used_meta":2460160,"kb_avail":752445392,"statfs":{"total":773094113280,"available":770504081408,"internally_reserved":0,"allocated":69779456,"data_stored":8818626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2519203840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-39114336,"num_objects":-10160,"num_object_clones":0,"num_object_copies":-20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-10309,"num_write_kb":-59123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-750,"num_bytes_recovered":-2915,"num_keys_recovered":0,"num_objects_omap":-22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001397"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-07T08:00:55.228158+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T08:00:55.228158+0000","last_peered":"2023-10-07T08:00:55.228158+0000","last_clean":"2023-10-07T08:00:55.228158+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T08:00:55.228158+0000","last_undegraded":"2023-10-07T08:00:55.228158+0000","last_fullsized":"2023-10-07T08:00:55.228158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320504,"kb_used_data":15024,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94051336,"statfs":{"total":96636764160,"available":96308568064,"internally_reserved":0,"allocated":15384576,"data_stored":5778374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63500000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67700000000000005}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.54700000000000004,"15min":0.55500000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63500000000000001,"5min":1.127,"15min":1.127},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58299999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76900000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.76300000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.626,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70599999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.80200000000000005}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.437,"15min":0.45400000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60299999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48799999999999999,"15min":0.51400000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.69199999999999995}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.50900000000000001,"15min":0.52500000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.53500000000000003,"15min":0.55600000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":1.048,"15min":1.048},"last":0.71099999999999997}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.53400000000000003,"15min":0.55500000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67700000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.751},{"interface":"front","average":{"1min":0.505,"5min":0.57999999999999996,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73799999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.752},{"interface":"front","average":{"1min":0.497,"5min":0.60099999999999998,"15min":0.61599999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.69699999999999995}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.75700000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.63600000000000001,"15min":0.68100000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66300000000000003,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.81399999999999995}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.69799999999999995,"15min":0.77200000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69099999999999995,"5min":2.653,"15min":2.653},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70999999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66500000000000004,"15min":0.72699999999999998},"min":{"1min":0.314,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67600000000000005,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48799999999999999,"15min":0.504},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.625,"5min":0.878,"15min":0.878},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64400000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.58499999999999996,"15min":0.60199999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66500000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56799999999999995}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.499},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319532,"kb_used_data":9956,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94052308,"statfs":{"total":96636764160,"available":96309563392,"internally_reserved":0,"allocated":10194944,"data_stored":855502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.443,"15min":0.46899999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53400000000000003,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.56699999999999995,"15min":0.63800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52600000000000002,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.66400000000000003,"15min":0.77600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78100000000000003,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.54400000000000004,"15min":0.57899999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71499999999999997,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.59099999999999997,"15min":0.64700000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70799999999999996,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.58799999999999997}]},{"osd":4,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65400000000000003,"15min":0.66700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":1.452,"15min":1.452},"last":0.67300000000000004}]},{"osd":5,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.996,"5min":3.3929999999999998,"15min":3.3929999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":1.546,"15min":1.546},"last":0.70699999999999996}]},{"osd":6,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.016,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.81299999999999994,"15min":0.87},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96799999999999997,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.79300000000000004}]},{"osd":7,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.82199999999999995,"15min":0.879},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0309999999999999,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.78500000000000003,"5min":1.585,"15min":1.585},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319648,"kb_used_data":10072,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94052192,"statfs":{"total":96636764160,"available":96309444608,"internally_reserved":0,"allocated":10313728,"data_stored":265792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.41199999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.70999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42999999999999999,"15min":0.44},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70099999999999996,"5min":1.298,"15min":1.298},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.73499999999999999}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.71099999999999997,"5min":1.032,"15min":1.032},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.141,"15min":1.141},"last":0.621}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88800000000000001,"5min":1.218,"15min":1.218},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.95499999999999996}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":0.69099999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82799999999999996,"5min":1.375,"15min":1.375},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86099999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.86399999999999999}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.401,"15min":1.401},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.69499999999999995,"15min":0.72299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90200000000000002,"5min":1.462,"15min":1.462},"last":0.97499999999999998}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.88},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83699999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.79200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320984,"kb_used_data":11408,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94050856,"statfs":{"total":96636764160,"available":96308076544,"internally_reserved":0,"allocated":11681792,"data_stored":267127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.39400000000000002,"15min":0.39600000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.45700000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.29599999999999999}]},{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.51100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67300000000000004}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.026,"5min":1.51,"15min":1.51},"last":0.81799999999999995}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.88,"15min":0.88},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.53000000000000003}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.68600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.84499999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.497}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.41699999999999998,"15min":0.437},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.53100000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42599999999999999,"15min":0.435},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64100000000000001,"5min":1.016,"15min":1.016},"last":0.32200000000000001}]},{"osd":1,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.53400000000000003,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.624,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.42399999999999999}]},{"osd":2,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.49399999999999999,"15min":0.51700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67900000000000005,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.60099999999999998,"5min":1.323,"15min":1.323},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93999999999999995,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96899999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.59599999999999997}]},{"osd":5,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.327},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.78800000000000003,"15min":0.83799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.751,"15min":0.78700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.78200000000000003,"15min":0.81999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.018,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.72099999999999997,"15min":0.74299999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.89300000000000002,"5min":1.528,"15min":1.528},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0369999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322136,"kb_used_data":12560,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94049704,"statfs":{"total":96636764160,"available":96306896896,"internally_reserved":0,"allocated":12861440,"data_stored":268261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":1.3959999999999999}]},{"osd":1,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.65400000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":1.1080000000000001}]},{"osd":2,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":1.425}]},{"osd":3,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78300000000000003}]},{"osd":5,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77400000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52400000000000002}]},{"osd":6,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.56399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":0.878},"last":0.56999999999999995}]},{"osd":7,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80600000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306132,"kb_used_data":7884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94065708,"statfs":{"total":96636764160,"available":96323284992,"internally_reserved":0,"allocated":8073216,"data_stored":859102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.60999999999999999,"15min":0.65600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58999999999999997,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.65400000000000003,"15min":0.70699999999999996},"min":{"1min":0.35599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":3.375,"15min":3.375},"last":0.41099999999999998}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.61699999999999999,"15min":0.66700000000000004},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64300000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.502},{"interface":"front","average":{"1min":0.442,"5min":0.62,"15min":0.66800000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.71499999999999997,"15min":0.79900000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":5.2699999999999996,"15min":5.2699999999999996},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89900000000000002,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.91500000000000004,"15min":1.0860000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":10.644,"15min":10.644},"last":0.57799999999999996}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.54300000000000004,"15min":0.58299999999999996},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57399999999999995,"5min":1.242,"15min":1.242},"last":0.81100000000000005}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.53600000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.64000000000000001,"15min":0.68899999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63600000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.59199999999999997,"15min":0.61399999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.75600000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65100000000000002,"5min":1.123,"15min":1.123},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T08:01:01.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T08:01:01.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.984+0000 7f712caff700 1 Processor -- start 2023-10-07T08:01:01.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.985+0000 7f712caff700 1 -- start start 2023-10-07T08:01:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712caff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7128155f20 0x7f71280c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712caff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712806b700 con 0x7f712806e9d0 2023-10-07T08:01:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712caff700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712806aaf0 con 0x7f71281562f0 2023-10-07T08:01:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712caff700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712806add0 con 0x7f7128155f20 2023-10-07T08:01:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f7126d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f712806e9d0 0x7f71280caf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47008/0 (socket says 172.21.15.7:47008) 2023-10-07T08:01:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f7126d9d700 1 -- 172.21.15.7:0/3278607354 learned_addr learned my addr 172.21.15.7:0/3278607354 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712659c700 1 --2- 172.21.15.7:0/3278607354 >> v2:172.21.15.172:3300/0 conn(0x7f7128155f20 0x7f71280c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712559a700 1 -- 172.21.15.7:0/3278607354 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115135251 0 0) 0x7f712806b700 con 0x7f712806e9d0 2023-10-07T08:01:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712559a700 1 -- 172.21.15.7:0/3278607354 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7110003570 con 0x7f712806e9d0 2023-10-07T08:01:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712559a700 1 -- 172.21.15.7:0/3278607354 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522969748 0 0) 0x7f712806aaf0 con 0x7f71281562f0 2023-10-07T08:01:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712559a700 1 -- 172.21.15.7:0/3278607354 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f712806b700 con 0x7f71281562f0 2023-10-07T08:01:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712659c700 1 -- 172.21.15.7:0/3278607354 >> v1:172.21.15.172:6789/0 conn(0x7f71281562f0 legacy=0x7f712806e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712659c700 1 -- 172.21.15.7:0/3278607354 >> v1:172.21.15.7:6789/0 conn(0x7f712806e9d0 legacy=0x7f71280caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712559a700 1 -- 172.21.15.7:0/3278607354 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 112738504 0 0) 0x7f7110003570 con 0x7f712806e9d0 2023-10-07T08:01:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.986+0000 7f712659c700 1 -- 172.21.15.7:0/3278607354 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712806b0b0 con 0x7f7128155f20 2023-10-07T08:01:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.987+0000 7f712659c700 1 --2- 172.21.15.7:0/3278607354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7128155f20 0x7f71280c92b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f71180098e0 tx=0x7f71180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=182d2d3a1b341b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.987+0000 7f712559a700 1 -- 172.21.15.7:0/3278607354 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200017f0 con 0x7f7128155f20 2023-10-07T08:01:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.987+0000 7f712559a700 1 -- 172.21.15.7:0/3278607354 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7118011420 con 0x7f7128155f20 2023-10-07T08:01:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.987+0000 7f712559a700 1 -- 172.21.15.7:0/3278607354 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71180117c0 con 0x7f7128155f20 2023-10-07T08:01:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.987+0000 7f712caff700 1 -- 172.21.15.7:0/3278607354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7128155f20 msgr2=0x7f71280c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.987+0000 7f712caff700 1 --2- 172.21.15.7:0/3278607354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7128155f20 0x7f71280c92b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f71180098e0 tx=0x7f71180091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.987+0000 7f712caff700 1 -- 172.21.15.7:0/3278607354 shutdown_connections 2023-10-07T08:01:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.987+0000 7f712caff700 1 --2- 172.21.15.7:0/3278607354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7128155f20 0x7f71280c92b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.987+0000 7f712caff700 1 -- 172.21.15.7:0/3278607354 >> 172.21.15.7:0/3278607354 conn(0x7f7128068bd0 msgr2=0x7f71280ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.988+0000 7f712caff700 1 -- 172.21.15.7:0/3278607354 shutdown_connections 2023-10-07T08:01:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.988+0000 7f712caff700 1 -- 172.21.15.7:0/3278607354 wait complete. 2023-10-07T08:01:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.989+0000 7f712caff700 1 Processor -- start 2023-10-07T08:01:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.989+0000 7f712caff700 1 -- start start 2023-10-07T08:01:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.989+0000 7f712caff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712806e9d0 0x7f71280c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f712caff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7128155f20 0x7f71281540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f712caff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71281562f0 0x7f71280c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f712caff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7128159380 con 0x7f71281562f0 2023-10-07T08:01:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f712caff700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71280c7350 con 0x7f712806e9d0 2023-10-07T08:01:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f712caff700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7128159590 con 0x7f7128155f20 2023-10-07T08:01:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7126d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71281562f0 0x7f71280c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7126d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71281562f0 0x7f71280c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45042/0 (socket says 172.21.15.7:45042) 2023-10-07T08:01:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7126d9d700 1 -- 172.21.15.7:0/1628902467 learned_addr learned my addr 172.21.15.7:0/1628902467 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f712659c700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712806e9d0 0x7f71280c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7125d9b700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7128155f20 0x7f71281540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7126d9d700 1 -- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7128155f20 msgr2=0x7f71281540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7126d9d700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7128155f20 0x7f71281540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7126d9d700 1 -- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712806e9d0 msgr2=0x7f71280c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7126d9d700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712806e9d0 0x7f71280c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7126d9d700 1 -- 172.21.15.7:0/1628902467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118009420 con 0x7f71281562f0 2023-10-07T08:01:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7125d9b700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7128155f20 0x7f71281540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:01:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f7126d9d700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71281562f0 0x7f71280c4060 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f712000cbd0 tx=0x7f7120007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f71167fc700 1 -- 172.21.15.7:0/1628902467 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120004dc0 con 0x7f71281562f0 2023-10-07T08:01:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.990+0000 7f71167fc700 1 -- 172.21.15.7:0/1628902467 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712000a600 con 0x7f71281562f0 2023-10-07T08:01:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.991+0000 7f71167fc700 1 -- 172.21.15.7:0/1628902467 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000a9c0 con 0x7f71281562f0 2023-10-07T08:01:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.991+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7128159870 con 0x7f71281562f0 2023-10-07T08:01:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.991+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71280c7060 con 0x7f71281562f0 2023-10-07T08:01:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.992+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7128159d90 con 0x7f71281562f0 2023-10-07T08:01:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.992+0000 7f71167fc700 1 -- 172.21.15.7:0/1628902467 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7120007150 con 0x7f71281562f0 2023-10-07T08:01:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.993+0000 7f71167fc700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7108066f30 0x7f71080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.993+0000 7f71167fc700 1 -- 172.21.15.7:0/1628902467 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7120083840 con 0x7f71281562f0 2023-10-07T08:01:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.993+0000 7f712659c700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7108066f30 0x7f71080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.993+0000 7f712659c700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7108066f30 0x7f71080693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71180093f0 tx=0x7f7118016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:01.997+0000 7f71167fc700 1 -- 172.21.15.7:0/1628902467 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7128159d90 con 0x7f71281562f0 2023-10-07T08:01:02.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.194+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7128159d90 con 0x7f7108066f30 2023-10-07T08:01:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.196+0000 7f71167fc700 1 -- 172.21.15.7:0/1628902467 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50762 (secure 0 0 0) 0x7f7128159d90 con 0x7f7108066f30 2023-10-07T08:01:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:01:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7108066f30 msgr2=0x7f71080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7108066f30 0x7f71080693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71180093f0 tx=0x7f7118016000 comp rx=0 tx=0).stop 2023-10-07T08:01:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71281562f0 msgr2=0x7f71280c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71281562f0 0x7f71280c4060 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f712000cbd0 tx=0x7f7120007d20 comp rx=0 tx=0).stop 2023-10-07T08:01:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 shutdown_connections 2023-10-07T08:01:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f7108066f30 0x7f71080693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71281562f0 0x7f71280c4060 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712806e9d0 0x7f71280c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 --2- 172.21.15.7:0/1628902467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7128155f20 0x7f71281540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 >> 172.21.15.7:0/1628902467 conn(0x7f7128068bd0 msgr2=0x7f7128154f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 shutdown_connections 2023-10-07T08:01:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.198+0000 7f712caff700 1 -- 172.21.15.7:0/1628902467 wait complete. 2023-10-07T08:01:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T08:01:02.213 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-10-07T08:01:02.070089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2529328,"kb_used_data":68144,"kb_used_omap":0,"kb_used_meta":2460160,"kb_avail":752445392,"statfs":{"total":773094113280,"available":770504081408,"internally_reserved":0,"allocated":69779456,"data_stored":8818626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2519203840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-39114336,"num_objects":-10160,"num_object_clones":0,"num_object_copies":-20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-10309,"num_write_kb":-59123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-750,"num_bytes_recovered":-2915,"num_keys_recovered":0,"num_objects_omap":-22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.637071"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-07T08:00:55.228158+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T08:00:55.228158+0000","last_peered":"2023-10-07T08:00:55.228158+0000","last_clean":"2023-10-07T08:00:55.228158+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T08:00:55.228158+0000","last_undegraded":"2023-10-07T08:00:55.228158+0000","last_fullsized":"2023-10-07T08:00:55.228158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320504,"kb_used_data":15024,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94051336,"statfs":{"total":96636764160,"available":96308568064,"internally_reserved":0,"allocated":15384576,"data_stored":5778374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63500000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67700000000000005}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.54700000000000004,"15min":0.55500000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63500000000000001,"5min":1.127,"15min":1.127},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58299999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76900000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.76300000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.626,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70599999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.80200000000000005}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.437,"15min":0.45400000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60299999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48799999999999999,"15min":0.51400000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.69199999999999995}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.50900000000000001,"15min":0.52500000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.53500000000000003,"15min":0.55600000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":1.048,"15min":1.048},"last":0.71099999999999997}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.53400000000000003,"15min":0.55500000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67700000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.751},{"interface":"front","average":{"1min":0.505,"5min":0.57999999999999996,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73799999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.752},{"interface":"front","average":{"1min":0.497,"5min":0.60099999999999998,"15min":0.61599999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.69699999999999995}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.75700000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.63600000000000001,"15min":0.68100000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66300000000000003,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.81399999999999995}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.69799999999999995,"15min":0.77200000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69099999999999995,"5min":2.653,"15min":2.653},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70999999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66500000000000004,"15min":0.72699999999999998},"min":{"1min":0.314,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67600000000000005,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48799999999999999,"15min":0.504},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.625,"5min":0.878,"15min":0.878},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64400000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.58499999999999996,"15min":0.60199999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66500000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56799999999999995}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.499},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319532,"kb_used_data":9956,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94052308,"statfs":{"total":96636764160,"available":96309563392,"internally_reserved":0,"allocated":10194944,"data_stored":855502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.443,"15min":0.46899999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53400000000000003,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.56699999999999995,"15min":0.63800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52600000000000002,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.66400000000000003,"15min":0.77600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78100000000000003,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.54400000000000004,"15min":0.57899999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71499999999999997,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.59099999999999997,"15min":0.64700000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70799999999999996,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.58799999999999997}]},{"osd":4,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65400000000000003,"15min":0.66700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":1.452,"15min":1.452},"last":0.67300000000000004}]},{"osd":5,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.996,"5min":3.3929999999999998,"15min":3.3929999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":1.546,"15min":1.546},"last":0.70699999999999996}]},{"osd":6,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.016,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.81299999999999994,"15min":0.87},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96799999999999997,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.79300000000000004}]},{"osd":7,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.82199999999999995,"15min":0.879},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0309999999999999,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.78500000000000003,"5min":1.585,"15min":1.585},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319648,"kb_used_data":10072,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94052192,"statfs":{"total":96636764160,"available":96309444608,"internally_reserved":0,"allocated":10313728,"data_stored":265792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.41199999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.70999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42999999999999999,"15min":0.44},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70099999999999996,"5min":1.298,"15min":1.298},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.73499999999999999}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.71099999999999997,"5min":1.032,"15min":1.032},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.141,"15min":1.141},"last":0.621}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88800000000000001,"5min":1.218,"15min":1.218},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.95499999999999996}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":0.69099999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82799999999999996,"5min":1.375,"15min":1.375},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86099999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.86399999999999999}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.401,"15min":1.401},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.69499999999999995,"15min":0.72299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90200000000000002,"5min":1.462,"15min":1.462},"last":0.97499999999999998}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.88},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83699999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.79200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320984,"kb_used_data":11408,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94050856,"statfs":{"total":96636764160,"available":96308076544,"internally_reserved":0,"allocated":11681792,"data_stored":267127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.39400000000000002,"15min":0.39600000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.45700000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.29599999999999999}]},{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.51100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67300000000000004}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.026,"5min":1.51,"15min":1.51},"last":0.81799999999999995}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.88,"15min":0.88},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.53000000000000003}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.68600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.84499999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.497}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.41699999999999998,"15min":0.437},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.53100000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42599999999999999,"15min":0.435},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64100000000000001,"5min":1.016,"15min":1.016},"last":0.32200000000000001}]},{"osd":1,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.53400000000000003,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.624,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.42399999999999999}]},{"osd":2,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.49399999999999999,"15min":0.51700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67900000000000005,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.60099999999999998,"5min":1.323,"15min":1.323},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93999999999999995,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96899999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.59599999999999997}]},{"osd":5,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.327},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.78800000000000003,"15min":0.83799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.751,"15min":0.78700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.78200000000000003,"15min":0.81999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.018,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.72099999999999997,"15min":0.74299999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.89300000000000002,"5min":1.528,"15min":1.528},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0369999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322136,"kb_used_data":12560,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94049704,"statfs":{"total":96636764160,"available":96306896896,"internally_reserved":0,"allocated":12861440,"data_stored":268261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":1.3959999999999999}]},{"osd":1,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.65400000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":1.1080000000000001}]},{"osd":2,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":1.425}]},{"osd":3,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78300000000000003}]},{"osd":5,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77400000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52400000000000002}]},{"osd":6,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.56399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":0.878},"last":0.56999999999999995}]},{"osd":7,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80600000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306132,"kb_used_data":7884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94065708,"statfs":{"total":96636764160,"available":96323284992,"internally_reserved":0,"allocated":8073216,"data_stored":859102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.60999999999999999,"15min":0.65600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58999999999999997,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.65400000000000003,"15min":0.70699999999999996},"min":{"1min":0.35599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":3.375,"15min":3.375},"last":0.41099999999999998}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.61699999999999999,"15min":0.66700000000000004},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64300000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.502},{"interface":"front","average":{"1min":0.442,"5min":0.62,"15min":0.66800000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.71499999999999997,"15min":0.79900000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":5.2699999999999996,"15min":5.2699999999999996},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89900000000000002,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.91500000000000004,"15min":1.0860000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":10.644,"15min":10.644},"last":0.57799999999999996}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.54300000000000004,"15min":0.58299999999999996},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57399999999999995,"5min":1.242,"15min":1.242},"last":0.81100000000000005}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.53600000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.64000000000000001,"15min":0.68899999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63600000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.59199999999999997,"15min":0.61399999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.75600000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65100000000000002,"5min":1.123,"15min":1.123},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T08:01:02.216 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-07T08:01:02.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T08:01:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.350+0000 7f43f60ef700 1 Processor -- start 2023-10-07T08:01:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.350+0000 7f43f60ef700 1 -- start start 2023-10-07T08:01:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43f60ef700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f01592c0 0x7f43f006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43f60ef700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43f00cb290 0x7f43f006a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43f60ef700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f006bb70 con 0x7f43f0159690 2023-10-07T08:01:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43f60ef700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f00cda10 con 0x7f43f00cb290 2023-10-07T08:01:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43f60ef700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f006af60 con 0x7f43f01592c0 2023-10-07T08:01:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43effff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43f00cb290 0x7f43f006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43effff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43f00cb290 0x7f43f006a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45044/0 (socket says 172.21.15.7:45044) 2023-10-07T08:01:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43effff700 1 -- 172.21.15.7:0/1479481488 learned_addr learned my addr 172.21.15.7:0/1479481488 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43ef7fe700 1 --2- 172.21.15.7:0/1479481488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f01592c0 0x7f43f006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43effff700 1 -- 172.21.15.7:0/1479481488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f01592c0 msgr2=0x7f43f006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43effff700 1 --2- 172.21.15.7:0/1479481488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f01592c0 0x7f43f006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43effff700 1 -- 172.21.15.7:0/1479481488 >> v1:172.21.15.172:6789/0 conn(0x7f43f0159690 legacy=0x7f43f00cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43effff700 1 -- 172.21.15.7:0/1479481488 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f006b240 con 0x7f43f00cb290 2023-10-07T08:01:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.351+0000 7f43ee7fc700 1 -- 172.21.15.7:0/1479481488 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073153777 0 0) 0x7f43f006bb70 con 0x7f43f0159690 2023-10-07T08:01:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43effff700 1 --2- 172.21.15.7:0/1479481488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f00cb290 0x7f43f006a930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009f20 tx=0x7f43e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6322dd3bff4b5547 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43ee7fc700 1 -- 172.21.15.7:0/1479481488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e800e070 con 0x7f43f00cb290 2023-10-07T08:01:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43ee7fc700 1 -- 172.21.15.7:0/1479481488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e800b9d0 con 0x7f43f00cb290 2023-10-07T08:01:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43ee7fc700 1 -- 172.21.15.7:0/1479481488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e80092e0 con 0x7f43f00cb290 2023-10-07T08:01:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43f60ef700 1 -- 172.21.15.7:0/1479481488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f00cb290 msgr2=0x7f43f006a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43f60ef700 1 --2- 172.21.15.7:0/1479481488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f00cb290 0x7f43f006a930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009f20 tx=0x7f43e8009840 comp rx=0 tx=0).stop 2023-10-07T08:01:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43f60ef700 1 -- 172.21.15.7:0/1479481488 shutdown_connections 2023-10-07T08:01:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43f60ef700 1 --2- 172.21.15.7:0/1479481488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f00cb290 0x7f43f006a930 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43f60ef700 1 --2- 172.21.15.7:0/1479481488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f01592c0 0x7f43f006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43f60ef700 1 -- 172.21.15.7:0/1479481488 >> 172.21.15.7:0/1479481488 conn(0x7f43f0068bd0 msgr2=0x7f43f00ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43f60ef700 1 -- 172.21.15.7:0/1479481488 shutdown_connections 2023-10-07T08:01:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.352+0000 7f43f60ef700 1 -- 172.21.15.7:0/1479481488 wait complete. 2023-10-07T08:01:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.354+0000 7f43f60ef700 1 Processor -- start 2023-10-07T08:01:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.354+0000 7f43f60ef700 1 -- start start 2023-10-07T08:01:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43f60ef700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f00cb290 0x7f43f0156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43f60ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f01592c0 0x7f43f015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43f60ef700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f0159690 0x7f43f015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43f60ef700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f01573e0 con 0x7f43f01592c0 2023-10-07T08:01:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43f60ef700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f015e350 con 0x7f43f0159690 2023-10-07T08:01:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43f60ef700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0159cb0 con 0x7f43f00cb290 2023-10-07T08:01:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f01592c0 0x7f43f015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f01592c0 0x7f43f015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45052/0 (socket says 172.21.15.7:45052) 2023-10-07T08:01:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43eeffd700 1 -- 172.21.15.7:0/2480367711 learned_addr learned my addr 172.21.15.7:0/2480367711 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43effff700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f0159690 0x7f43f015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43ef7fe700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f00cb290 0x7f43f0156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43eeffd700 1 -- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f00cb290 msgr2=0x7f43f0156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43eeffd700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f00cb290 0x7f43f0156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43eeffd700 1 -- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f0159690 msgr2=0x7f43f015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43eeffd700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f0159690 0x7f43f015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43eeffd700 1 -- 172.21.15.7:0/2480367711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8009ae0 con 0x7f43f01592c0 2023-10-07T08:01:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43eeffd700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f01592c0 0x7f43f015d060 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f43e400c7e0 tx=0x7f43e4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.355+0000 7f43d77fe700 1 -- 172.21.15.7:0/2480367711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40105c0 con 0x7f43f01592c0 2023-10-07T08:01:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.356+0000 7f43d77fe700 1 -- 172.21.15.7:0/2480367711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e4010740 con 0x7f43f01592c0 2023-10-07T08:01:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.356+0000 7f43d77fe700 1 -- 172.21.15.7:0/2480367711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4010b00 con 0x7f43f01592c0 2023-10-07T08:01:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.356+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f015dab0 con 0x7f43f01592c0 2023-10-07T08:01:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.356+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f015e070 con 0x7f43f01592c0 2023-10-07T08:01:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.357+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f0159e20 con 0x7f43f01592c0 2023-10-07T08:01:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.357+0000 7f43d77fe700 1 -- 172.21.15.7:0/2480367711 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43e401b3b0 con 0x7f43f01592c0 2023-10-07T08:01:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.358+0000 7f43d77fe700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.358+0000 7f43d77fe700 1 -- 172.21.15.7:0/2480367711 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f43e4083880 con 0x7f43f01592c0 2023-10-07T08:01:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.358+0000 7f43ef7fe700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.358+0000 7f43ef7fe700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43dc066f30 0x7f43dc0693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009db0 tx=0x7f43e00091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.362+0000 7f43d77fe700 1 -- 172.21.15.7:0/2480367711 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43e4050d50 con 0x7f43f01592c0 2023-10-07T08:01:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.561+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43f00716c0 con 0x7f43dc066f30 2023-10-07T08:01:02.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.564+0000 7f43d77fe700 1 -- 172.21.15.7:0/2480367711 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50762 (secure 0 0 0) 0x7f43f00716c0 con 0x7f43dc066f30 2023-10-07T08:01:02.569 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:01:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43dc066f30 msgr2=0x7f43dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43dc066f30 0x7f43dc0693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009db0 tx=0x7f43e00091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f01592c0 msgr2=0x7f43f015d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f01592c0 0x7f43f015d060 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f43e400c7e0 tx=0x7f43e4007df0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 shutdown_connections 2023-10-07T08:01:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f0159690 0x7f43f015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43f00cb290 0x7f43f0156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 --2- 172.21.15.7:0/2480367711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f01592c0 0x7f43f015d060 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 >> 172.21.15.7:0/2480367711 conn(0x7f43f0068bd0 msgr2=0x7f43f00c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.566+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 shutdown_connections 2023-10-07T08:01:02.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.567+0000 7f43f60ef700 1 -- 172.21.15.7:0/2480367711 wait complete. 2023-10-07T08:01:02.573 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T08:01:02.581 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-10-07T08:01:02.070089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2529328,"kb_used_data":68144,"kb_used_omap":0,"kb_used_meta":2460160,"kb_avail":752445392,"statfs":{"total":773094113280,"available":770504081408,"internally_reserved":0,"allocated":69779456,"data_stored":8818626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2519203840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-39114336,"num_objects":-10160,"num_object_clones":0,"num_object_copies":-20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-10309,"num_write_kb":-59123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-750,"num_bytes_recovered":-2915,"num_keys_recovered":0,"num_objects_omap":-22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.637071"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-07T08:00:55.228158+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T08:00:55.228158+0000","last_peered":"2023-10-07T08:00:55.228158+0000","last_clean":"2023-10-07T08:00:55.228158+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T08:00:55.228158+0000","last_undegraded":"2023-10-07T08:00:55.228158+0000","last_fullsized":"2023-10-07T08:00:55.228158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320504,"kb_used_data":15024,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94051336,"statfs":{"total":96636764160,"available":96308568064,"internally_reserved":0,"allocated":15384576,"data_stored":5778374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63500000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67700000000000005}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.54700000000000004,"15min":0.55500000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63500000000000001,"5min":1.127,"15min":1.127},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58299999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76900000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.76300000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.626,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70599999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.80200000000000005}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.437,"15min":0.45400000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60299999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48799999999999999,"15min":0.51400000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.69199999999999995}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.50900000000000001,"15min":0.52500000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.53500000000000003,"15min":0.55600000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":1.048,"15min":1.048},"last":0.71099999999999997}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.53400000000000003,"15min":0.55500000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67700000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.751},{"interface":"front","average":{"1min":0.505,"5min":0.57999999999999996,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73799999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.752},{"interface":"front","average":{"1min":0.497,"5min":0.60099999999999998,"15min":0.61599999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.69699999999999995}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.75700000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.63600000000000001,"15min":0.68100000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66300000000000003,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.81399999999999995}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.69799999999999995,"15min":0.77200000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69099999999999995,"5min":2.653,"15min":2.653},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70999999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66500000000000004,"15min":0.72699999999999998},"min":{"1min":0.314,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67600000000000005,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48799999999999999,"15min":0.504},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.625,"5min":0.878,"15min":0.878},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64400000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.58499999999999996,"15min":0.60199999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66500000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56799999999999995}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.499},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319532,"kb_used_data":9956,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94052308,"statfs":{"total":96636764160,"available":96309563392,"internally_reserved":0,"allocated":10194944,"data_stored":855502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.443,"15min":0.46899999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53400000000000003,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.56699999999999995,"15min":0.63800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52600000000000002,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.66400000000000003,"15min":0.77600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78100000000000003,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.54400000000000004,"15min":0.57899999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71499999999999997,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.59099999999999997,"15min":0.64700000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70799999999999996,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.58799999999999997}]},{"osd":4,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65400000000000003,"15min":0.66700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":1.452,"15min":1.452},"last":0.67300000000000004}]},{"osd":5,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.996,"5min":3.3929999999999998,"15min":3.3929999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":1.546,"15min":1.546},"last":0.70699999999999996}]},{"osd":6,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.016,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.81299999999999994,"15min":0.87},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96799999999999997,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.79300000000000004}]},{"osd":7,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.82199999999999995,"15min":0.879},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0309999999999999,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.78500000000000003,"5min":1.585,"15min":1.585},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319648,"kb_used_data":10072,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94052192,"statfs":{"total":96636764160,"available":96309444608,"internally_reserved":0,"allocated":10313728,"data_stored":265792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.41199999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.70999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42999999999999999,"15min":0.44},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70099999999999996,"5min":1.298,"15min":1.298},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.73499999999999999}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.71099999999999997,"5min":1.032,"15min":1.032},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.141,"15min":1.141},"last":0.621}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88800000000000001,"5min":1.218,"15min":1.218},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.95499999999999996}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":0.69099999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82799999999999996,"5min":1.375,"15min":1.375},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86099999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.86399999999999999}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.401,"15min":1.401},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.69499999999999995,"15min":0.72299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90200000000000002,"5min":1.462,"15min":1.462},"last":0.97499999999999998}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.88},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83699999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.79200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320984,"kb_used_data":11408,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94050856,"statfs":{"total":96636764160,"available":96308076544,"internally_reserved":0,"allocated":11681792,"data_stored":267127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.39400000000000002,"15min":0.39600000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.45700000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.29599999999999999}]},{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.51100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67300000000000004}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.026,"5min":1.51,"15min":1.51},"last":0.81799999999999995}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.88,"15min":0.88},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.53000000000000003}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.68600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.84499999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.497}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.41699999999999998,"15min":0.437},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.53100000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42599999999999999,"15min":0.435},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64100000000000001,"5min":1.016,"15min":1.016},"last":0.32200000000000001}]},{"osd":1,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.53400000000000003,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.624,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.42399999999999999}]},{"osd":2,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.49399999999999999,"15min":0.51700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67900000000000005,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.60099999999999998,"5min":1.323,"15min":1.323},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93999999999999995,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96899999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.59599999999999997}]},{"osd":5,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.327},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.78800000000000003,"15min":0.83799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.751,"15min":0.78700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.78200000000000003,"15min":0.81999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.018,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.72099999999999997,"15min":0.74299999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.89300000000000002,"5min":1.528,"15min":1.528},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0369999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322136,"kb_used_data":12560,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94049704,"statfs":{"total":96636764160,"available":96306896896,"internally_reserved":0,"allocated":12861440,"data_stored":268261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":1.3959999999999999}]},{"osd":1,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.65400000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":1.1080000000000001}]},{"osd":2,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":1.425}]},{"osd":3,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78300000000000003}]},{"osd":5,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77400000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52400000000000002}]},{"osd":6,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.56399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":0.878},"last":0.56999999999999995}]},{"osd":7,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80600000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306132,"kb_used_data":7884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94065708,"statfs":{"total":96636764160,"available":96323284992,"internally_reserved":0,"allocated":8073216,"data_stored":859102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.60999999999999999,"15min":0.65600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58999999999999997,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.65400000000000003,"15min":0.70699999999999996},"min":{"1min":0.35599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":3.375,"15min":3.375},"last":0.41099999999999998}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.61699999999999999,"15min":0.66700000000000004},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64300000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.502},{"interface":"front","average":{"1min":0.442,"5min":0.62,"15min":0.66800000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.71499999999999997,"15min":0.79900000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":5.2699999999999996,"15min":5.2699999999999996},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89900000000000002,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.91500000000000004,"15min":1.0860000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":10.644,"15min":10.644},"last":0.57799999999999996}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.54300000000000004,"15min":0.58299999999999996},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57399999999999995,"5min":1.242,"15min":1.242},"last":0.81100000000000005}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.53600000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.64000000000000001,"15min":0.68899999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63600000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.59199999999999997,"15min":0.61399999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.75600000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65100000000000002,"5min":1.123,"15min":1.123},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T08:01:02.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T08:01:02.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.716+0000 7f401e7ec700 1 Processor -- start 2023-10-07T08:01:02.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.716+0000 7f401e7ec700 1 -- start start 2023-10-07T08:01:02.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401e7ec700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4018159310 0x7f40180c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401e7ec700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40181596e0 0x7f401806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401e7ec700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4018068100 con 0x7f40180ced30 2023-10-07T08:01:02.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401e7ec700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4018068350 con 0x7f4018159310 2023-10-07T08:01:02.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401e7ec700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4018068630 con 0x7f40181596e0 2023-10-07T08:01:02.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401cd89700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40181596e0 0x7f401806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f4017fff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4018159310 0x7f40180c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401cd89700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40181596e0 0x7f401806b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:36836/0 (socket says 172.21.15.7:36836) 2023-10-07T08:01:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401cd89700 1 -- 172.21.15.7:0/2761685974 learned_addr learned my addr 172.21.15.7:0/2761685974 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401cd89700 1 -- 172.21.15.7:0/2761685974 >> v2:172.21.15.172:3300/0 conn(0x7f4018159310 msgr2=0x7f40180c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401cd89700 1 --2- 172.21.15.7:0/2761685974 >> v2:172.21.15.172:3300/0 conn(0x7f4018159310 0x7f40180c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401cd89700 1 -- 172.21.15.7:0/2761685974 >> v1:172.21.15.172:6789/0 conn(0x7f40180ced30 legacy=0x7f40180c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f401cd89700 1 -- 172.21.15.7:0/2761685974 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018068910 con 0x7f40181596e0 2023-10-07T08:01:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f4016ffd700 1 -- 172.21.15.7:0/2761685974 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000053919 0 0) 0x7f4018068100 con 0x7f40180ced30 2023-10-07T08:01:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.717+0000 7f4017fff700 1 --2- 172.21.15.7:0/2761685974 >> v2:172.21.15.172:3300/0 conn(0x7f4018159310 0x7f40180c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f401cd89700 1 --2- 172.21.15.7:0/2761685974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40181596e0 0x7f401806b620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4010009f20 tx=0x7f4010009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c5589a8162c79ee server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f4016ffd700 1 -- 172.21.15.7:0/2761685974 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401000e070 con 0x7f40181596e0 2023-10-07T08:01:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f4016ffd700 1 -- 172.21.15.7:0/2761685974 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401000b9d0 con 0x7f40181596e0 2023-10-07T08:01:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f4016ffd700 1 -- 172.21.15.7:0/2761685974 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40100092e0 con 0x7f40181596e0 2023-10-07T08:01:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f401e7ec700 1 -- 172.21.15.7:0/2761685974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40181596e0 msgr2=0x7f401806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f401e7ec700 1 --2- 172.21.15.7:0/2761685974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40181596e0 0x7f401806b620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4010009f20 tx=0x7f4010009840 comp rx=0 tx=0).stop 2023-10-07T08:01:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f401e7ec700 1 -- 172.21.15.7:0/2761685974 shutdown_connections 2023-10-07T08:01:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f401e7ec700 1 --2- 172.21.15.7:0/2761685974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40181596e0 0x7f401806b620 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f401e7ec700 1 --2- 172.21.15.7:0/2761685974 >> v2:172.21.15.172:3300/0 conn(0x7f4018159310 0x7f40180c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f401e7ec700 1 -- 172.21.15.7:0/2761685974 >> 172.21.15.7:0/2761685974 conn(0x7f4018067ac0 msgr2=0x7f40180ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f401e7ec700 1 -- 172.21.15.7:0/2761685974 shutdown_connections 2023-10-07T08:01:02.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.718+0000 7f401e7ec700 1 -- 172.21.15.7:0/2761685974 wait complete. 2023-10-07T08:01:02.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.720+0000 7f401e7ec700 1 Processor -- start 2023-10-07T08:01:02.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.720+0000 7f401e7ec700 1 -- start start 2023-10-07T08:01:02.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f401e7ec700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4018159310 0x7f401815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f401e7ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40181596e0 0x7f401813ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f401e7ec700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f401806d220 0x7f401806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f401e7ec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401806dbd0 con 0x7f40181596e0 2023-10-07T08:01:02.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f401e7ec700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401815c820 con 0x7f4018159310 2023-10-07T08:01:02.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f401e7ec700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40180b30f0 con 0x7f401806d220 2023-10-07T08:01:02.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f40177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40181596e0 0x7f401813ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f40177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40181596e0 0x7f401813ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45066/0 (socket says 172.21.15.7:45066) 2023-10-07T08:01:02.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f40177fe700 1 -- 172.21.15.7:0/1925298344 learned_addr learned my addr 172.21.15.7:0/1925298344 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:02.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f401cd89700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f401806d220 0x7f401806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f4017fff700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4018159310 0x7f401815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f40177fe700 1 -- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f401806d220 msgr2=0x7f401806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f40177fe700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f401806d220 0x7f401806d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f40177fe700 1 -- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4018159310 msgr2=0x7f401815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f40177fe700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4018159310 0x7f401815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f40177fe700 1 -- 172.21.15.7:0/1925298344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4008009420 con 0x7f40181596e0 2023-10-07T08:01:02.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f401cd89700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f401806d220 0x7f401806d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:02.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.721+0000 7f40177fe700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40181596e0 0x7f401813ba20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f400c00c7e0 tx=0x7f400c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:02.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.722+0000 7f3ffffff700 1 -- 172.21.15.7:0/1925298344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c011590 con 0x7f40181596e0 2023-10-07T08:01:02.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.722+0000 7f3ffffff700 1 -- 172.21.15.7:0/1925298344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c011710 con 0x7f40181596e0 2023-10-07T08:01:02.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.722+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4010009ae0 con 0x7f40181596e0 2023-10-07T08:01:02.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.722+0000 7f3ffffff700 1 -- 172.21.15.7:0/1925298344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c011ad0 con 0x7f40181596e0 2023-10-07T08:01:02.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.722+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40180b24e0 con 0x7f40181596e0 2023-10-07T08:01:02.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.723+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40180c57a0 con 0x7f40181596e0 2023-10-07T08:01:02.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.723+0000 7f3ffffff700 1 -- 172.21.15.7:0/1925298344 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f400c01b3b0 con 0x7f40181596e0 2023-10-07T08:01:02.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.724+0000 7f3ffffff700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4004066f30 0x7f40040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:02.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.724+0000 7f3ffffff700 1 -- 172.21.15.7:0/1925298344 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f400c083690 con 0x7f40181596e0 2023-10-07T08:01:02.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.724+0000 7f4017fff700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4004066f30 0x7f40040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:02.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.724+0000 7f4017fff700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4004066f30 0x7f40040693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4008009f40 tx=0x7f40080091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:02.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.728+0000 7f3ffffff700 1 -- 172.21.15.7:0/1925298344 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f400c050c00 con 0x7f40181596e0 2023-10-07T08:01:02.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.934+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40180716c0 con 0x7f40181596e0 2023-10-07T08:01:02.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.934+0000 7f3ffffff700 1 -- 172.21.15.7:0/1925298344 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12307 (secure 0 0 0) 0x7f400c017020 con 0x7f40181596e0 2023-10-07T08:01:02.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:01:02.936 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"fsid":"c71d63e6-6de9-4139-83e7-b0bb7f6375d7","created":"2023-10-07T07:57:32.470891+0000","modified":"2023-10-07T08:00:57.430557+0000","last_up_change":"2023-10-07T07:57:42.649003+0000","last_in_change":"2023-10-07T07:57:35.016789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T07:57:44.081562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"26598042-733a-424c-a5e5-3b75a67a460e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":120000},{"type":"v1","addr":"172.21.15.7:6811","nonce":120000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":120000},{"type":"v1","addr":"172.21.15.7:6815","nonce":120000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":120000},{"type":"v1","addr":"172.21.15.7:6823","nonce":120000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":120000},{"type":"v1","addr":"172.21.15.7:6819","nonce":120000}]},"public_addr":"172.21.15.7:6811/120000","cluster_addr":"172.21.15.7:6815/120000","heartbeat_back_addr":"172.21.15.7:6823/120000","heartbeat_front_addr":"172.21.15.7:6819/120000","state":["exists","up"]},{"osd":1,"uuid":"26aad281-717c-4bde-8dca-873b8714ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":120002},{"type":"v1","addr":"172.21.15.7:6810","nonce":120002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":120002},{"type":"v1","addr":"172.21.15.7:6814","nonce":120002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":120002},{"type":"v1","addr":"172.21.15.7:6822","nonce":120002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":120002},{"type":"v1","addr":"172.21.15.7:6818","nonce":120002}]},"public_addr":"172.21.15.7:6810/120002","cluster_addr":"172.21.15.7:6814/120002","heartbeat_back_addr":"172.21.15.7:6822/120002","heartbeat_front_addr":"172.21.15.7:6818/120002","state":["exists","up"]},{"osd":2,"uuid":"7b622d63-3165-4b67-8545-fb0d62cc728b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":120011},{"type":"v1","addr":"172.21.15.7:6825","nonce":120011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":120011},{"type":"v1","addr":"172.21.15.7:6827","nonce":120011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":120011},{"type":"v1","addr":"172.21.15.7:6831","nonce":120011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":120011},{"type":"v1","addr":"172.21.15.7:6829","nonce":120011}]},"public_addr":"172.21.15.7:6825/120011","cluster_addr":"172.21.15.7:6827/120011","heartbeat_back_addr":"172.21.15.7:6831/120011","heartbeat_front_addr":"172.21.15.7:6829/120011","state":["exists","up"]},{"osd":3,"uuid":"6998b22c-4c0e-432c-89fb-c1f8311be875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":120001},{"type":"v1","addr":"172.21.15.7:6801","nonce":120001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":120001},{"type":"v1","addr":"172.21.15.7:6803","nonce":120001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":120001},{"type":"v1","addr":"172.21.15.7:6807","nonce":120001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":120001},{"type":"v1","addr":"172.21.15.7:6805","nonce":120001}]},"public_addr":"172.21.15.7:6801/120001","cluster_addr":"172.21.15.7:6803/120001","heartbeat_back_addr":"172.21.15.7:6807/120001","heartbeat_front_addr":"172.21.15.7:6805/120001","state":["exists","up"]},{"osd":4,"uuid":"8347d98c-5559-43e5-83e2-e053778e801f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":120243},{"type":"v1","addr":"172.21.15.172:6801","nonce":120243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":120243},{"type":"v1","addr":"172.21.15.172:6803","nonce":120243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":120243},{"type":"v1","addr":"172.21.15.172:6807","nonce":120243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":120243},{"type":"v1","addr":"172.21.15.172:6805","nonce":120243}]},"public_addr":"172.21.15.172:6801/120243","cluster_addr":"172.21.15.172:6803/120243","heartbeat_back_addr":"172.21.15.172:6807/120243","heartbeat_front_addr":"172.21.15.172:6805/120243","state":["exists","up"]},{"osd":5,"uuid":"189820da-0b58-4f63-9405-647998088464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":120244},{"type":"v1","addr":"172.21.15.172:6809","nonce":120244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":120244},{"type":"v1","addr":"172.21.15.172:6811","nonce":120244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":120244},{"type":"v1","addr":"172.21.15.172:6815","nonce":120244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":120244},{"type":"v1","addr":"172.21.15.172:6813","nonce":120244}]},"public_addr":"172.21.15.172:6809/120244","cluster_addr":"172.21.15.172:6811/120244","heartbeat_back_addr":"172.21.15.172:6815/120244","heartbeat_front_addr":"172.21.15.172:6813/120244","state":["exists","up"]},{"osd":6,"uuid":"070138d2-5051-4c4a-bfa3-8afbdcbb0806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":120273},{"type":"v1","addr":"172.21.15.172:6817","nonce":120273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":120273},{"type":"v1","addr":"172.21.15.172:6819","nonce":120273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":120273},{"type":"v1","addr":"172.21.15.172:6823","nonce":120273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":120273},{"type":"v1","addr":"172.21.15.172:6821","nonce":120273}]},"public_addr":"172.21.15.172:6817/120273","cluster_addr":"172.21.15.172:6819/120273","heartbeat_back_addr":"172.21.15.172:6823/120273","heartbeat_front_addr":"172.21.15.172:6821/120273","state":["exists","up"]},{"osd":7,"uuid":"585b3d12-8bca-4a10-9d87-9bb9a95311b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":120274},{"type":"v1","addr":"172.21.15.172:6825","nonce":120274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":120274},{"type":"v1","addr":"172.21.15.172:6827","nonce":120274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":120274},{"type":"v1","addr":"172.21.15.172:6831","nonce":120274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":120274},{"type":"v1","addr":"172.21.15.172:6829","nonce":120274}]},"public_addr":"172.21.15.172:6825/120274","cluster_addr":"172.21.15.172:6827/120274","heartbeat_back_addr":"172.21.15.172:6831/120274","heartbeat_front_addr":"172.21.15.172:6829/120274","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:41.653377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.218155+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.192518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.102156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:39.884403+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.187290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:40.203311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T07:57:42.278575+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/763397529":"2023-10-08T08:00:54.423343+0000","172.21.15.7:6834/763397529":"2023-10-08T08:00:54.423343+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T08:01:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4004066f30 msgr2=0x7f40040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4004066f30 0x7f40040693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4008009f40 tx=0x7f40080091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40181596e0 msgr2=0x7f401813ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40181596e0 0x7f401813ba20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f400c00c7e0 tx=0x7f400c007df0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 shutdown_connections 2023-10-07T08:01:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f4004066f30 0x7f40040693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f401806d220 0x7f401806d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4018159310 0x7f401815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 --2- 172.21.15.7:0/1925298344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40181596e0 0x7f401813ba20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 >> 172.21.15.7:0/1925298344 conn(0x7f4018067ac0 msgr2=0x7f40180cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 shutdown_connections 2023-10-07T08:01:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:02.937+0000 7f401e7ec700 1 -- 172.21.15.7:0/1925298344 wait complete. 2023-10-07T08:01:03.952 INFO:tasks.ceph:Scrubbing osd.0 2023-10-07T08:01:03.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-07T08:01:04.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.097+0000 7f919a1dd700 1 Processor -- start 2023-10-07T08:01:04.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.097+0000 7f919a1dd700 1 -- start start 2023-10-07T08:01:04.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f919a1dd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f919406bc70 0x7f9194153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f919a1dd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919413c860 con 0x7f91941535c0 2023-10-07T08:01:04.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f919a1dd700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919406a110 con 0x7f919415d2b0 2023-10-07T08:01:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f919a1dd700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919415de40 con 0x7f919406bc70 2023-10-07T08:01:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f91937fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91941535c0 0x7f919406b560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47022/0 (socket says 172.21.15.7:47022) 2023-10-07T08:01:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f91937fe700 1 -- 172.21.15.7:0/142697 learned_addr learned my addr 172.21.15.7:0/142697 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f9192ffd700 1 --2- 172.21.15.7:0/142697 >> v2:172.21.15.172:3300/0 conn(0x7f919406bc70 0x7f9194153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493569743 0 0) 0x7f919413c860 con 0x7f91941535c0 2023-10-07T08:01:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9180003570 con 0x7f91941535c0 2023-10-07T08:01:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.098+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3311019749 0 0) 0x7f9180003570 con 0x7f91941535c0 2023-10-07T08:01:04.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 >> v2:172.21.15.172:3300/0 conn(0x7f919406bc70 msgr2=0x7f9194153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:04.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 --2- 172.21.15.7:0/142697 >> v2:172.21.15.172:3300/0 conn(0x7f919406bc70 0x7f9194153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 >> v1:172.21.15.172:6789/0 conn(0x7f919415d2b0 legacy=0x7f919415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9194159cb0 con 0x7f91941535c0 2023-10-07T08:01:04.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 6033801 0 0) 0x7f917c0015f0 con 0x7f91941535c0 2023-10-07T08:01:04.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 >> v1:172.21.15.7:6789/0 conn(0x7f91941535c0 legacy=0x7f919406b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9180003a60 0x7f9180003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9180004810 0x7f9180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91800051c0 0x7f918000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919415de40 con 0x7f9180004810 2023-10-07T08:01:04.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918000c050 con 0x7f91800051c0 2023-10-07T08:01:04.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918000c210 con 0x7f9180003a60 2023-10-07T08:01:04.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f9192ffd700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9180004810 0x7f9180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f91937fe700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9180003a60 0x7f9180003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f9193fff700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91800051c0 0x7f918000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f9192ffd700 1 -- 172.21.15.7:0/142697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9180003a60 msgr2=0x7f9180003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f9192ffd700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9180003a60 0x7f9180003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f9192ffd700 1 -- 172.21.15.7:0/142697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91800051c0 msgr2=0x7f918000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f9192ffd700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91800051c0 0x7f918000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f9193fff700 1 -- 172.21.15.7:0/142697 reap_dead start 2023-10-07T08:01:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f9192ffd700 1 -- 172.21.15.7:0/142697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9180003570 con 0x7f9180004810 2023-10-07T08:01:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.099+0000 7f9192ffd700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9180004810 0x7f9180004c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f919406c070 tx=0x7f91840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc52b375c1c11711 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918400e710 con 0x7f9180004810 2023-10-07T08:01:04.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918400e870 con 0x7f9180004810 2023-10-07T08:01:04.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f91927fc700 1 -- 172.21.15.7:0/142697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918400ec30 con 0x7f9180004810 2023-10-07T08:01:04.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f919a1dd700 1 -- 172.21.15.7:0/142697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9180004810 msgr2=0x7f9180004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f919a1dd700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9180004810 0x7f9180004c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f919406c070 tx=0x7f91840091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f919a1dd700 1 -- 172.21.15.7:0/142697 shutdown_connections 2023-10-07T08:01:04.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f919a1dd700 1 --2- 172.21.15.7:0/142697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9180004810 0x7f9180004c80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f919a1dd700 1 -- 172.21.15.7:0/142697 >> 172.21.15.7:0/142697 conn(0x7f9194067ac0 msgr2=0x7f91940ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:04.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f919a1dd700 1 -- 172.21.15.7:0/142697 shutdown_connections 2023-10-07T08:01:04.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.100+0000 7f919a1dd700 1 -- 172.21.15.7:0/142697 wait complete. 2023-10-07T08:01:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.102+0000 7f919a1dd700 1 Processor -- start 2023-10-07T08:01:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.102+0000 7f919a1dd700 1 -- start start 2023-10-07T08:01:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.102+0000 7f919a1dd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9180004810 0x7f9194167bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.102+0000 7f919a1dd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f919416b160 0x7f919416a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.102+0000 7f919a1dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f919413b740 0x7f919413a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.102+0000 7f919a1dd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919416c200 con 0x7f919413b740 2023-10-07T08:01:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.102+0000 7f919a1dd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919416af00 con 0x7f919416b160 2023-10-07T08:01:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.102+0000 7f919a1dd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9194158730 con 0x7f9180004810 2023-10-07T08:01:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f919413b740 0x7f919413a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f919413b740 0x7f919413a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45082/0 (socket says 172.21.15.7:45082) 2023-10-07T08:01:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9193fff700 1 -- 172.21.15.7:0/169714199 learned_addr learned my addr 172.21.15.7:0/169714199 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f91937fe700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9180004810 0x7f9194167bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9192ffd700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f919416b160 0x7f919416a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9193fff700 1 -- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9180004810 msgr2=0x7f9194167bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9193fff700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9180004810 0x7f9194167bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9193fff700 1 -- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f919416b160 msgr2=0x7f919416a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9193fff700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f919416b160 0x7f919416a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9193fff700 1 -- 172.21.15.7:0/169714199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919416b5f0 con 0x7f919413b740 2023-10-07T08:01:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f91937fe700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9180004810 0x7f9194167bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:01:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9192ffd700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f919416b160 0x7f919416a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:01:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f9193fff700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f919413b740 0x7f919413a1b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9194067fd0 tx=0x7f918c0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f918b7fe700 1 -- 172.21.15.7:0/169714199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c00c820 con 0x7f919413b740 2023-10-07T08:01:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f918b7fe700 1 -- 172.21.15.7:0/169714199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c00c9a0 con 0x7f919413b740 2023-10-07T08:01:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f918b7fe700 1 -- 172.21.15.7:0/169714199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c00cd60 con 0x7f919413b740 2023-10-07T08:01:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919416b8d0 con 0x7f919413b740 2023-10-07T08:01:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.103+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919416be90 con 0x7f919413b740 2023-10-07T08:01:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.104+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f91940716c0 con 0x7f919413b740 2023-10-07T08:01:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.105+0000 7f918b7fe700 1 -- 172.21.15.7:0/169714199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f918c013630 con 0x7f919413b740 2023-10-07T08:01:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.105+0000 7f918b7fe700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9174066f30 0x7f91740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.105+0000 7f918b7fe700 1 -- 172.21.15.7:0/169714199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f918c084080 con 0x7f919413b740 2023-10-07T08:01:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.105+0000 7f918b7fe700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f917406cd80 0x7f917406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.105+0000 7f918b7fe700 1 -- 172.21.15.7:0/169714199 --> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f917406f860 con 0x7f917406cd80 2023-10-07T08:01:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.105+0000 7f918b7fe700 1 -- 172.21.15.7:0/169714199 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f918c0866c0 con 0x7f919413b740 2023-10-07T08:01:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.105+0000 7f91937fe700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9174066f30 0x7f91740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.106+0000 7f9192ffd700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f917406cd80 0x7f917406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.106+0000 7f9192ffd700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f917406cd80 0x7f917406f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.106+0000 7f91937fe700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9174066f30 0x7f91740693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f917c0011d0 tx=0x7f917c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.107+0000 7f918b7fe700 1 -- 172.21.15.7:0/169714199 <== osd.0 v2:172.21.15.7:6809/120000 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f917406f860 con 0x7f917406cd80 2023-10-07T08:01:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.121+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 --> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9194159f90 con 0x7f917406cd80 2023-10-07T08:01:04.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.121+0000 7f918b7fe700 1 -- 172.21.15.7:0/169714199 <== osd.0 v2:172.21.15.7:6809/120000 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9194159f90 con 0x7f917406cd80 2023-10-07T08:01:04.123 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T08:01:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T08:01:04.123 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T08:01:04.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f917406cd80 msgr2=0x7f917406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.7:6809/120000,v1:172.21.15.7:6811/120000] conn(0x7f917406cd80 0x7f917406f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9174066f30 msgr2=0x7f91740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f9174066f30 0x7f91740693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f917c0011d0 tx=0x7f917c009040 comp rx=0 tx=0).stop 2023-10-07T08:01:04.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f919413b740 msgr2=0x7f919413a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 --2- 172.21.15.7:0/169714199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f919413b740 0x7f919413a1b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9194067fd0 tx=0x7f918c0016d0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f9193fff700 1 -- 172.21.15.7:0/169714199 reap_dead start 2023-10-07T08:01:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 shutdown_connections 2023-10-07T08:01:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 >> 172.21.15.7:0/169714199 conn(0x7f9194067ac0 msgr2=0x7f919415b300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 shutdown_connections 2023-10-07T08:01:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.122+0000 7f919a1dd700 1 -- 172.21.15.7:0/169714199 wait complete. 2023-10-07T08:01:04.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-07T08:01:04.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.268+0000 7fdca48ba700 1 Processor -- start 2023-10-07T08:01:04.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.268+0000 7fdca48ba700 1 -- start start 2023-10-07T08:01:04.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.269+0000 7fdca48ba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdca01570a0 0x7fdca00c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.269+0000 7fdca48ba700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdca0157470 0x7fdca006b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.269+0000 7fdca48ba700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 0x7fdca006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.269+0000 7fdca48ba700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca006c000 con 0x7fdca01570a0 2023-10-07T08:01:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.269+0000 7fdca48ba700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca00c3b80 con 0x7fdca0157470 2023-10-07T08:01:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.269+0000 7fdca48ba700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00c2f70 con 0x7fdca006bb60 2023-10-07T08:01:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.269+0000 7fdc9e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdca01570a0 0x7fdca00c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.269+0000 7fdc9e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdca01570a0 0x7fdca00c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45086/0 (socket says 172.21.15.7:45086) 2023-10-07T08:01:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9e59c700 1 -- 172.21.15.7:0/2147469221 learned_addr learned my addr 172.21.15.7:0/2147469221 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9dd9b700 1 --2- 172.21.15.7:0/2147469221 >> v2:172.21.15.172:3300/0 conn(0x7fdca0157470 0x7fdca006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9ed9d700 1 --2- 172.21.15.7:0/2147469221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 0x7fdca006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9e59c700 1 -- 172.21.15.7:0/2147469221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 msgr2=0x7fdca006cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9e59c700 1 --2- 172.21.15.7:0/2147469221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 0x7fdca006cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9e59c700 1 -- 172.21.15.7:0/2147469221 >> v2:172.21.15.172:3300/0 conn(0x7fdca0157470 msgr2=0x7fdca006b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9e59c700 1 --2- 172.21.15.7:0/2147469221 >> v2:172.21.15.172:3300/0 conn(0x7fdca0157470 0x7fdca006b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9e59c700 1 -- 172.21.15.7:0/2147469221 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca00c3250 con 0x7fdca01570a0 2023-10-07T08:01:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9dd9b700 1 --2- 172.21.15.7:0/2147469221 >> v2:172.21.15.172:3300/0 conn(0x7fdca0157470 0x7fdca006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:01:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9ed9d700 1 --2- 172.21.15.7:0/2147469221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 0x7fdca006cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:01:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9e59c700 1 --2- 172.21.15.7:0/2147469221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 0x7fdca00c8800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdca006df20 tx=0x7fdc94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0561241844d80f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9d59a700 1 -- 172.21.15.7:0/2147469221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400b860 con 0x7fdca01570a0 2023-10-07T08:01:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9d59a700 1 -- 172.21.15.7:0/2147469221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9400b9c0 con 0x7fdca01570a0 2023-10-07T08:01:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.270+0000 7fdc9d59a700 1 -- 172.21.15.7:0/2147469221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400bd60 con 0x7fdca01570a0 2023-10-07T08:01:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.271+0000 7fdca48ba700 1 -- 172.21.15.7:0/2147469221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 msgr2=0x7fdca00c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.271+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2147469221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 0x7fdca00c8800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdca006df20 tx=0x7fdc94009250 comp rx=0 tx=0).stop 2023-10-07T08:01:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.271+0000 7fdca48ba700 1 -- 172.21.15.7:0/2147469221 shutdown_connections 2023-10-07T08:01:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.271+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2147469221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 0x7fdca006cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.271+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2147469221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 0x7fdca00c8800 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.271+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2147469221 >> v2:172.21.15.172:3300/0 conn(0x7fdca0157470 0x7fdca006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.271+0000 7fdca48ba700 1 -- 172.21.15.7:0/2147469221 >> 172.21.15.7:0/2147469221 conn(0x7fdca0067ac0 msgr2=0x7fdca00ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.271+0000 7fdca48ba700 1 -- 172.21.15.7:0/2147469221 shutdown_connections 2023-10-07T08:01:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.271+0000 7fdca48ba700 1 -- 172.21.15.7:0/2147469221 wait complete. 2023-10-07T08:01:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.272+0000 7fdca48ba700 1 Processor -- start 2023-10-07T08:01:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.272+0000 7fdca48ba700 1 -- start start 2023-10-07T08:01:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdca48ba700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 0x7fdca00c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdca48ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 0x7fdca00b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdca48ba700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0157470 0x7fdca00b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdca48ba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00c84a0 con 0x7fdca01570a0 2023-10-07T08:01:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdca48ba700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00b1d30 con 0x7fdca0157470 2023-10-07T08:01:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdca48ba700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00b2b60 con 0x7fdca006bb60 2023-10-07T08:01:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 0x7fdca00b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 0x7fdca00b17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45094/0 (socket says 172.21.15.7:45094) 2023-10-07T08:01:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9dd9b700 1 -- 172.21.15.7:0/2943783578 learned_addr learned my addr 172.21.15.7:0/2943783578 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9e59c700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 0x7fdca00c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9ed9d700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0157470 0x7fdca00b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9dd9b700 1 -- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 msgr2=0x7fdca00c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9dd9b700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 0x7fdca00c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9dd9b700 1 -- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0157470 msgr2=0x7fdca00b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9dd9b700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0157470 0x7fdca00b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.273+0000 7fdc9dd9b700 1 -- 172.21.15.7:0/2943783578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940094f0 con 0x7fdca01570a0 2023-10-07T08:01:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.274+0000 7fdc9dd9b700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 0x7fdca00b17f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c880 tx=0x7fdc90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.274+0000 7fdc8e7fc700 1 -- 172.21.15.7:0/2943783578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90011600 con 0x7fdca01570a0 2023-10-07T08:01:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.274+0000 7fdc8e7fc700 1 -- 172.21.15.7:0/2943783578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc90011760 con 0x7fdca01570a0 2023-10-07T08:01:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.274+0000 7fdc8e7fc700 1 -- 172.21.15.7:0/2943783578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90011b00 con 0x7fdca01570a0 2023-10-07T08:01:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.274+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca00b1f50 con 0x7fdca01570a0 2023-10-07T08:01:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.274+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca00b2590 con 0x7fdca01570a0 2023-10-07T08:01:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.275+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca00b2910 con 0x7fdca01570a0 2023-10-07T08:01:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.275+0000 7fdc8e7fc700 1 -- 172.21.15.7:0/2943783578 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc90018350 con 0x7fdca01570a0 2023-10-07T08:01:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.276+0000 7fdc8e7fc700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdc80066f30 0x7fdc800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.276+0000 7fdc8e7fc700 1 -- 172.21.15.7:0/2943783578 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdc90084100 con 0x7fdca01570a0 2023-10-07T08:01:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.276+0000 7fdc9e59c700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdc80066f30 0x7fdc800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.276+0000 7fdc9e59c700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdc80066f30 0x7fdc800693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdca006df20 tx=0x7fdc94017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.280+0000 7fdc8e7fc700 1 -- 172.21.15.7:0/2943783578 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc900515d0 con 0x7fdca01570a0 2023-10-07T08:01:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.479+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fdca006b970 con 0x7fdc80066f30 2023-10-07T08:01:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.480+0000 7fdc8e7fc700 1 -- 172.21.15.7:0/2943783578 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdca006b970 con 0x7fdc80066f30 2023-10-07T08:01:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdc80066f30 msgr2=0x7fdc800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdc80066f30 0x7fdc800693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdca006df20 tx=0x7fdc94017000 comp rx=0 tx=0).stop 2023-10-07T08:01:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 msgr2=0x7fdca00b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 0x7fdca00b17f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c880 tx=0x7fdc90008040 comp rx=0 tx=0).stop 2023-10-07T08:01:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 shutdown_connections 2023-10-07T08:01:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdc80066f30 0x7fdc800693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0157470 0x7fdca00b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca006bb60 0x7fdca00c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 --2- 172.21.15.7:0/2943783578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdca01570a0 0x7fdca00b17f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 >> 172.21.15.7:0/2943783578 conn(0x7fdca0067ac0 msgr2=0x7fdca0152940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 shutdown_connections 2023-10-07T08:01:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.482+0000 7fdca48ba700 1 -- 172.21.15.7:0/2943783578 wait complete. 2023-10-07T08:01:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2023-10-07T08:01:04.497 INFO:tasks.ceph:Scrubbing osd.1 2023-10-07T08:01:04.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-07T08:01:04.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.633+0000 7f8604a2a700 1 Processor -- start 2023-10-07T08:01:04.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.633+0000 7f8604a2a700 1 -- start start 2023-10-07T08:01:04.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f8604a2a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86001598c0 0x7f860006af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f8604a2a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86000a9920 con 0x7f86001594f0 2023-10-07T08:01:04.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f8604a2a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860006b4f0 con 0x7f86000c1d30 2023-10-07T08:01:04.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f8604a2a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f860006b7d0 con 0x7f86001598c0 2023-10-07T08:01:04.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f85fdd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86001598c0 0x7f860006af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f85fe59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f86001594f0 0x7f86000674f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47024/0 (socket says 172.21.15.7:47024) 2023-10-07T08:01:04.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f85fe59c700 1 -- 172.21.15.7:0/3906689603 learned_addr learned my addr 172.21.15.7:0/3906689603 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:04.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f85fdd9b700 1 -- 172.21.15.7:0/3906689603 >> v1:172.21.15.172:6789/0 conn(0x7f86000c1d30 legacy=0x7f86000a9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:04.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f85fdd9b700 1 -- 172.21.15.7:0/3906689603 >> v1:172.21.15.7:6789/0 conn(0x7f86001594f0 legacy=0x7f86000674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f85fdd9b700 1 -- 172.21.15.7:0/3906689603 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860006bab0 con 0x7f86001598c0 2023-10-07T08:01:04.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.634+0000 7f85fd59a700 1 -- 172.21.15.7:0/3906689603 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007818108 0 0) 0x7f86000a9920 con 0x7f86001594f0 2023-10-07T08:01:04.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.635+0000 7f85fdd9b700 1 --2- 172.21.15.7:0/3906689603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 0x7f860006af10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f85f00098e0 tx=0x7f85f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18211645e8237d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.635+0000 7f85fd59a700 1 -- 172.21.15.7:0/3906689603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0009e80 con 0x7f86001598c0 2023-10-07T08:01:04.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.635+0000 7f85fd59a700 1 -- 172.21.15.7:0/3906689603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f000b820 con 0x7f86001598c0 2023-10-07T08:01:04.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.635+0000 7f85fd59a700 1 -- 172.21.15.7:0/3906689603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f000bba0 con 0x7f86001598c0 2023-10-07T08:01:04.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.635+0000 7f8604a2a700 1 -- 172.21.15.7:0/3906689603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 msgr2=0x7f860006af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.635+0000 7f8604a2a700 1 --2- 172.21.15.7:0/3906689603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 0x7f860006af10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f85f00098e0 tx=0x7f85f00091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.635+0000 7f8604a2a700 1 -- 172.21.15.7:0/3906689603 shutdown_connections 2023-10-07T08:01:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.635+0000 7f8604a2a700 1 --2- 172.21.15.7:0/3906689603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 0x7f860006af10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.635+0000 7f8604a2a700 1 -- 172.21.15.7:0/3906689603 >> 172.21.15.7:0/3906689603 conn(0x7f86000cda60 msgr2=0x7f8600068100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.636+0000 7f8604a2a700 1 -- 172.21.15.7:0/3906689603 shutdown_connections 2023-10-07T08:01:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.636+0000 7f8604a2a700 1 -- 172.21.15.7:0/3906689603 wait complete. 2023-10-07T08:01:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.637+0000 7f8604a2a700 1 Processor -- start 2023-10-07T08:01:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.637+0000 7f8604a2a700 1 -- start start 2023-10-07T08:01:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f8604a2a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86000c1d30 0x7f8600149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f8604a2a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86001594f0 0x7f86000caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f8604a2a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 0x7f86000cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f8604a2a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86000afd30 con 0x7f86001598c0 2023-10-07T08:01:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f8604a2a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86000b08b0 con 0x7f86000c1d30 2023-10-07T08:01:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f8604a2a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86000aff80 con 0x7f86001594f0 2023-10-07T08:01:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 0x7f86000cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 0x7f86000cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45106/0 (socket says 172.21.15.7:45106) 2023-10-07T08:01:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fed9d700 1 -- 172.21.15.7:0/217145585 learned_addr learned my addr 172.21.15.7:0/217145585 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fe59c700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86000c1d30 0x7f8600149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fdd9b700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86001594f0 0x7f86000caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fed9d700 1 -- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86001594f0 msgr2=0x7f86000caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fed9d700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86001594f0 0x7f86000caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fed9d700 1 -- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86000c1d30 msgr2=0x7f8600149950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fed9d700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86000c1d30 0x7f8600149950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fed9d700 1 -- 172.21.15.7:0/217145585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f0009420 con 0x7f86001598c0 2023-10-07T08:01:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85fed9d700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 0x7f86000cb4a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c930 tx=0x7f85f80026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85ee7fc700 1 -- 172.21.15.7:0/217145585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8012630 con 0x7f86001598c0 2023-10-07T08:01:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f85ee7fc700 1 -- 172.21.15.7:0/217145585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f80127d0 con 0x7f86001598c0 2023-10-07T08:01:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.638+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86000b0260 con 0x7f86001598c0 2023-10-07T08:01:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.639+0000 7f85ee7fc700 1 -- 172.21.15.7:0/217145585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8012b90 con 0x7f86001598c0 2023-10-07T08:01:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.639+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8600149e90 con 0x7f86001598c0 2023-10-07T08:01:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.639+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86000716c0 con 0x7f86001598c0 2023-10-07T08:01:04.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.640+0000 7f85ee7fc700 1 -- 172.21.15.7:0/217145585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f85f801a3b0 con 0x7f86001598c0 2023-10-07T08:01:04.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.640+0000 7f85ee7fc700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f85e0066f30 0x7f85e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.640+0000 7f85ee7fc700 1 -- 172.21.15.7:0/217145585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f85f8083e80 con 0x7f86001598c0 2023-10-07T08:01:04.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.641+0000 7f85ee7fc700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7f85e006cd80 0x7f85e006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.641+0000 7f85ee7fc700 1 -- 172.21.15.7:0/217145585 --> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85e006f860 con 0x7f85e006cd80 2023-10-07T08:01:04.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.641+0000 7f85ee7fc700 1 -- 172.21.15.7:0/217145585 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f80864c0 con 0x7f86001598c0 2023-10-07T08:01:04.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.641+0000 7f85fe59c700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f85e0066f30 0x7f85e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.641+0000 7f85fdd9b700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7f85e006cd80 0x7f85e006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.641+0000 7f85fdd9b700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7f85e006cd80 0x7f85e006f190 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.641+0000 7f85fe59c700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f85e0066f30 0x7f85e00693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85f4007fd0 tx=0x7f85f4007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.642+0000 7f85ee7fc700 1 -- 172.21.15.7:0/217145585 <== osd.1 v2:172.21.15.7:6808/120002 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f85e006f860 con 0x7f85e006cd80 2023-10-07T08:01:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.656+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 --> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f86000694a0 con 0x7f85e006cd80 2023-10-07T08:01:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f85ee7fc700 1 -- 172.21.15.7:0/217145585 <== osd.1 v2:172.21.15.7:6808/120002 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f86000694a0 con 0x7f85e006cd80 2023-10-07T08:01:04.658 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T08:01:04.658 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T08:01:04.658 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T08:01:04.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7f85e006cd80 msgr2=0x7f85e006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.7:6808/120002,v1:172.21.15.7:6810/120002] conn(0x7f85e006cd80 0x7f85e006f190 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f85e0066f30 msgr2=0x7f85e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f85e0066f30 0x7f85e00693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85f4007fd0 tx=0x7f85f4007d30 comp rx=0 tx=0).stop 2023-10-07T08:01:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 msgr2=0x7f86000cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 --2- 172.21.15.7:0/217145585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86001598c0 0x7f86000cb4a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c930 tx=0x7f85f80026e0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f85fed9d700 1 -- 172.21.15.7:0/217145585 reap_dead start 2023-10-07T08:01:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 shutdown_connections 2023-10-07T08:01:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 >> 172.21.15.7:0/217145585 conn(0x7f86000cda60 msgr2=0x7f86000cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 shutdown_connections 2023-10-07T08:01:04.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.657+0000 7f8604a2a700 1 -- 172.21.15.7:0/217145585 wait complete. 2023-10-07T08:01:04.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-07T08:01:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.804+0000 7f6305aa3700 1 Processor -- start 2023-10-07T08:01:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.805+0000 7f6305aa3700 1 -- start start 2023-10-07T08:01:04.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f6305aa3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f630006ed10 0x7f6300066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f6305aa3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300067530 0x7f630006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f6305aa3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630006e2c0 con 0x7f6300140a80 2023-10-07T08:01:04.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f6305aa3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6300067920 con 0x7f630006ed10 2023-10-07T08:01:04.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f6305aa3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63000c7940 con 0x7f6300067530 2023-10-07T08:01:04.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f62feffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f630006ed10 0x7f6300066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f62feffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f630006ed10 0x7f6300066ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45116/0 (socket says 172.21.15.7:45116) 2023-10-07T08:01:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f62feffd700 1 -- 172.21.15.7:0/4242195547 learned_addr learned my addr 172.21.15.7:0/4242195547 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f62fffff700 1 --2- 172.21.15.7:0/4242195547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300067530 0x7f630006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f62feffd700 1 -- 172.21.15.7:0/4242195547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300067530 msgr2=0x7f630006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f62feffd700 1 --2- 172.21.15.7:0/4242195547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300067530 0x7f630006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f62feffd700 1 -- 172.21.15.7:0/4242195547 >> v1:172.21.15.172:6789/0 conn(0x7f6300140a80 legacy=0x7f63001582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.806+0000 7f62feffd700 1 -- 172.21.15.7:0/4242195547 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000c7c20 con 0x7f630006ed10 2023-10-07T08:01:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f62feffd700 1 --2- 172.21.15.7:0/4242195547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630006ed10 0x7f6300066ff0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f62f40099b0 tx=0x7f62f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1ddd7932faa4ec server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f62fe7fc700 1 -- 172.21.15.7:0/4242195547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f400b860 con 0x7f630006ed10 2023-10-07T08:01:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f62fe7fc700 1 -- 172.21.15.7:0/4242195547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f400b9c0 con 0x7f630006ed10 2023-10-07T08:01:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f62fe7fc700 1 -- 172.21.15.7:0/4242195547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f400bd60 con 0x7f630006ed10 2023-10-07T08:01:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f6305aa3700 1 -- 172.21.15.7:0/4242195547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630006ed10 msgr2=0x7f6300066ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f6305aa3700 1 --2- 172.21.15.7:0/4242195547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630006ed10 0x7f6300066ff0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f62f40099b0 tx=0x7f62f4009250 comp rx=0 tx=0).stop 2023-10-07T08:01:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f6305aa3700 1 -- 172.21.15.7:0/4242195547 shutdown_connections 2023-10-07T08:01:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f6305aa3700 1 --2- 172.21.15.7:0/4242195547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300067530 0x7f630006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f6305aa3700 1 --2- 172.21.15.7:0/4242195547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630006ed10 0x7f6300066ff0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.807+0000 7f6305aa3700 1 -- 172.21.15.7:0/4242195547 >> 172.21.15.7:0/4242195547 conn(0x7f6300068bd0 msgr2=0x7f63000ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.808+0000 7f6305aa3700 1 -- 172.21.15.7:0/4242195547 shutdown_connections 2023-10-07T08:01:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.808+0000 7f6305aa3700 1 -- 172.21.15.7:0/4242195547 wait complete. 2023-10-07T08:01:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.809+0000 7f6305aa3700 1 Processor -- start 2023-10-07T08:01:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.809+0000 7f6305aa3700 1 -- start start 2023-10-07T08:01:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f6305aa3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300067530 0x7f63000c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f6305aa3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f630006ed10 0x7f630013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f6305aa3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300140a80 0x7f630015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f6305aa3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630015bdf0 con 0x7f6300067530 2023-10-07T08:01:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f6305aa3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630015c0d0 con 0x7f630006ed10 2023-10-07T08:01:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f6305aa3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630015c3b0 con 0x7f6300140a80 2023-10-07T08:01:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62ff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300067530 0x7f63000c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62ff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300067530 0x7f63000c4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45124/0 (socket says 172.21.15.7:45124) 2023-10-07T08:01:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62ff7fe700 1 -- 172.21.15.7:0/2180952325 learned_addr learned my addr 172.21.15.7:0/2180952325 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62feffd700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f630006ed10 0x7f630013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62fffff700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300140a80 0x7f630015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62ff7fe700 1 -- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300140a80 msgr2=0x7f630015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62ff7fe700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300140a80 0x7f630015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62ff7fe700 1 -- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f630006ed10 msgr2=0x7f630013f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62ff7fe700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f630006ed10 0x7f630013f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62ff7fe700 1 -- 172.21.15.7:0/2180952325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40094f0 con 0x7f6300067530 2023-10-07T08:01:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62feffd700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f630006ed10 0x7f630013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:01:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62fffff700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300140a80 0x7f630015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:01:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62ff7fe700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300067530 0x7f63000c4370 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c4e0 tx=0x7f62f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62e77fe700 1 -- 172.21.15.7:0/2180952325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0011590 con 0x7f6300067530 2023-10-07T08:01:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62e77fe700 1 -- 172.21.15.7:0/2180952325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f0011710 con 0x7f6300067530 2023-10-07T08:01:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630015c690 con 0x7f6300067530 2023-10-07T08:01:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.810+0000 7f62e77fe700 1 -- 172.21.15.7:0/2180952325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0011ad0 con 0x7f6300067530 2023-10-07T08:01:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.811+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63000c48b0 con 0x7f6300067530 2023-10-07T08:01:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.811+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630015cbb0 con 0x7f6300067530 2023-10-07T08:01:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.812+0000 7f62e77fe700 1 -- 172.21.15.7:0/2180952325 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62f001b3b0 con 0x7f6300067530 2023-10-07T08:01:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.812+0000 7f62e77fe700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f62ec066f30 0x7f62ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.812+0000 7f62e77fe700 1 -- 172.21.15.7:0/2180952325 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f62f0083760 con 0x7f6300067530 2023-10-07T08:01:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.813+0000 7f62feffd700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f62ec066f30 0x7f62ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.813+0000 7f62feffd700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f62ec066f30 0x7f62ec0693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62f40099b0 tx=0x7f62f4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:04.817+0000 7f62e77fe700 1 -- 172.21.15.7:0/2180952325 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f630015cbb0 con 0x7f6300067530 2023-10-07T08:01:05.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.017+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f630015cbb0 con 0x7f62ec066f30 2023-10-07T08:01:05.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.018+0000 7f62e77fe700 1 -- 172.21.15.7:0/2180952325 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f630015cbb0 con 0x7f62ec066f30 2023-10-07T08:01:05.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f62ec066f30 msgr2=0x7f62ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f62ec066f30 0x7f62ec0693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62f40099b0 tx=0x7f62f4017000 comp rx=0 tx=0).stop 2023-10-07T08:01:05.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300067530 msgr2=0x7f63000c4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300067530 0x7f63000c4370 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c4e0 tx=0x7f62f000c8a0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 shutdown_connections 2023-10-07T08:01:05.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f62ec066f30 0x7f62ec0693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6300140a80 0x7f630015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300067530 0x7f63000c4370 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 --2- 172.21.15.7:0/2180952325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f630006ed10 0x7f630013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 >> 172.21.15.7:0/2180952325 conn(0x7f6300068bd0 msgr2=0x7f6300159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:05.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 shutdown_connections 2023-10-07T08:01:05.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.020+0000 7f6305aa3700 1 -- 172.21.15.7:0/2180952325 wait complete. 2023-10-07T08:01:05.025 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2023-10-07T08:01:05.035 INFO:tasks.ceph:Scrubbing osd.2 2023-10-07T08:01:05.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-07T08:01:05.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.179+0000 7f3b2d214700 1 Processor -- start 2023-10-07T08:01:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.179+0000 7f3b2d214700 1 -- start start 2023-10-07T08:01:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.180+0000 7f3b2d214700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3b2813c580 0x7f3b2815a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.180+0000 7f3b2d214700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2815dac0 con 0x7f3b2813c1b0 2023-10-07T08:01:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.180+0000 7f3b2d214700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2815df50 con 0x7f3b2806b4f0 2023-10-07T08:01:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.180+0000 7f3b2d214700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2815d190 con 0x7f3b2813c580 2023-10-07T08:01:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.180+0000 7f3b26d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3b2813c1b0 0x7f3b280c2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47040/0 (socket says 172.21.15.7:47040) 2023-10-07T08:01:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.180+0000 7f3b26d9d700 1 -- 172.21.15.7:0/1417099122 learned_addr learned my addr 172.21.15.7:0/1417099122 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.180+0000 7f3b2659c700 1 --2- 172.21.15.7:0/1417099122 >> v2:172.21.15.172:3300/0 conn(0x7f3b2813c580 0x7f3b2815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.181+0000 7f3b25d9b700 1 -- 172.21.15.7:0/1417099122 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215659377 0 0) 0x7f3b2815dac0 con 0x7f3b2813c1b0 2023-10-07T08:01:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.181+0000 7f3b25d9b700 1 -- 172.21.15.7:0/1417099122 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b14003570 con 0x7f3b2813c1b0 2023-10-07T08:01:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.181+0000 7f3b2659c700 1 -- 172.21.15.7:0/1417099122 >> v1:172.21.15.172:6789/0 conn(0x7f3b2806b4f0 legacy=0x7f3b28163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.181+0000 7f3b2659c700 1 -- 172.21.15.7:0/1417099122 >> v1:172.21.15.7:6789/0 conn(0x7f3b2813c1b0 legacy=0x7f3b280c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.181+0000 7f3b2659c700 1 -- 172.21.15.7:0/1417099122 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2815d470 con 0x7f3b2813c580 2023-10-07T08:01:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.181+0000 7f3b2659c700 1 --2- 172.21.15.7:0/1417099122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2813c580 0x7f3b2815a650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3b180098e0 tx=0x7f3b180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc09758971e4c870 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.181+0000 7f3b25d9b700 1 -- 172.21.15.7:0/1417099122 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800b640 con 0x7f3b2813c580 2023-10-07T08:01:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.181+0000 7f3b25d9b700 1 -- 172.21.15.7:0/1417099122 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1800b7a0 con 0x7f3b2813c580 2023-10-07T08:01:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.181+0000 7f3b25d9b700 1 -- 172.21.15.7:0/1417099122 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800bb40 con 0x7f3b2813c580 2023-10-07T08:01:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.182+0000 7f3b2d214700 1 -- 172.21.15.7:0/1417099122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2813c580 msgr2=0x7f3b2815a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.182+0000 7f3b2d214700 1 --2- 172.21.15.7:0/1417099122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2813c580 0x7f3b2815a650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3b180098e0 tx=0x7f3b180091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.182+0000 7f3b2d214700 1 -- 172.21.15.7:0/1417099122 shutdown_connections 2023-10-07T08:01:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.182+0000 7f3b2d214700 1 --2- 172.21.15.7:0/1417099122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2813c580 0x7f3b2815a650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.182+0000 7f3b2d214700 1 -- 172.21.15.7:0/1417099122 >> 172.21.15.7:0/1417099122 conn(0x7f3b28067a70 msgr2=0x7f3b280ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.182+0000 7f3b2d214700 1 -- 172.21.15.7:0/1417099122 shutdown_connections 2023-10-07T08:01:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.182+0000 7f3b2d214700 1 -- 172.21.15.7:0/1417099122 wait complete. 2023-10-07T08:01:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.183+0000 7f3b2d214700 1 Processor -- start 2023-10-07T08:01:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.183+0000 7f3b2d214700 1 -- start start 2023-10-07T08:01:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b2d214700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b2806b4f0 0x7f3b280b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b2d214700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b2813c1b0 0x7f3b2816d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b2d214700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2813c580 0x7f3b2813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b2d214700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2816a8b0 con 0x7f3b2806b4f0 2023-10-07T08:01:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b2d214700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280b3d80 con 0x7f3b2813c580 2023-10-07T08:01:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b2d214700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b28169af0 con 0x7f3b2813c1b0 2023-10-07T08:01:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b26d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b2806b4f0 0x7f3b280b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b26d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b2806b4f0 0x7f3b280b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45140/0 (socket says 172.21.15.7:45140) 2023-10-07T08:01:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b26d9d700 1 -- 172.21.15.7:0/177455573 learned_addr learned my addr 172.21.15.7:0/177455573 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b2759e700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2813c580 0x7f3b2813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b2659c700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b2813c1b0 0x7f3b2816d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b26d9d700 1 -- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b2813c1b0 msgr2=0x7f3b2816d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b26d9d700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b2813c1b0 0x7f3b2816d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b26d9d700 1 -- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2813c580 msgr2=0x7f3b2813b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b26d9d700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2813c580 0x7f3b2813b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b26d9d700 1 -- 172.21.15.7:0/177455573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b18009420 con 0x7f3b2806b4f0 2023-10-07T08:01:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.184+0000 7f3b2759e700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2813c580 0x7f3b2813b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:01:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.185+0000 7f3b26d9d700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b2806b4f0 0x7f3b280b36a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c7c0 tx=0x7f3b10007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.185+0000 7f3b1effd700 1 -- 172.21.15.7:0/177455573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10005b30 con 0x7f3b2806b4f0 2023-10-07T08:01:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.185+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b28169dd0 con 0x7f3b2806b4f0 2023-10-07T08:01:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.185+0000 7f3b1effd700 1 -- 172.21.15.7:0/177455573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10005c90 con 0x7f3b2806b4f0 2023-10-07T08:01:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.185+0000 7f3b1effd700 1 -- 172.21.15.7:0/177455573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10007340 con 0x7f3b2806b4f0 2023-10-07T08:01:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.185+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2816a390 con 0x7f3b2806b4f0 2023-10-07T08:01:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.186+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b280716c0 con 0x7f3b2806b4f0 2023-10-07T08:01:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.186+0000 7f3b1effd700 1 -- 172.21.15.7:0/177455573 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b1001b400 con 0x7f3b2806b4f0 2023-10-07T08:01:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.187+0000 7f3b1effd700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3b08066f30 0x7f3b080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.187+0000 7f3b1effd700 1 -- 172.21.15.7:0/177455573 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3b10083e70 con 0x7f3b2806b4f0 2023-10-07T08:01:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.187+0000 7f3b1effd700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7f3b0806cd80 0x7f3b0806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.187+0000 7f3b1effd700 1 -- 172.21.15.7:0/177455573 --> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b0806f860 con 0x7f3b0806cd80 2023-10-07T08:01:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.187+0000 7f3b1effd700 1 -- 172.21.15.7:0/177455573 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b100863d0 con 0x7f3b2806b4f0 2023-10-07T08:01:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.187+0000 7f3b2659c700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3b08066f30 0x7f3b080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.187+0000 7f3b2759e700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7f3b0806cd80 0x7f3b0806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.187+0000 7f3b2759e700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7f3b0806cd80 0x7f3b0806f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.188+0000 7f3b2659c700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3b08066f30 0x7f3b080693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3b2806f500 tx=0x7f3b18017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.188+0000 7f3b1effd700 1 -- 172.21.15.7:0/177455573 <== osd.2 v2:172.21.15.7:6824/120011 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3b0806f860 con 0x7f3b0806cd80 2023-10-07T08:01:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.203+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 --> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3b280cebf0 con 0x7f3b0806cd80 2023-10-07T08:01:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.203+0000 7f3b1effd700 1 -- 172.21.15.7:0/177455573 <== osd.2 v2:172.21.15.7:6824/120011 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3b280cebf0 con 0x7f3b0806cd80 2023-10-07T08:01:05.204 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T08:01:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T08:01:05.205 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T08:01:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.203+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7f3b0806cd80 msgr2=0x7f3b0806f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.203+0000 7f3b2d214700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.7:6824/120011,v1:172.21.15.7:6825/120011] conn(0x7f3b0806cd80 0x7f3b0806f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.204+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3b08066f30 msgr2=0x7f3b080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.204+0000 7f3b2d214700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3b08066f30 0x7f3b080693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3b2806f500 tx=0x7f3b18017000 comp rx=0 tx=0).stop 2023-10-07T08:01:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.204+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b2806b4f0 msgr2=0x7f3b280b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.204+0000 7f3b2d214700 1 --2- 172.21.15.7:0/177455573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b2806b4f0 0x7f3b280b36a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c7c0 tx=0x7f3b10007e00 comp rx=0 tx=0).stop 2023-10-07T08:01:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.204+0000 7f3b2759e700 1 -- 172.21.15.7:0/177455573 reap_dead start 2023-10-07T08:01:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.204+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 shutdown_connections 2023-10-07T08:01:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.204+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 >> 172.21.15.7:0/177455573 conn(0x7f3b28067a70 msgr2=0x7f3b28147e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.204+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 shutdown_connections 2023-10-07T08:01:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.204+0000 7f3b2d214700 1 -- 172.21.15.7:0/177455573 wait complete. 2023-10-07T08:01:05.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-07T08:01:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.355+0000 7f38a2c00700 1 Processor -- start 2023-10-07T08:01:05.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.355+0000 7f38a2c00700 1 -- start start 2023-10-07T08:01:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a2c00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f389c0c9af0 0x7f389c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a2c00700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 0x7f389c06e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a2c00700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c15a330 con 0x7f389c063230 2023-10-07T08:01:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a2c00700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389c0b03a0 con 0x7f389c0c9af0 2023-10-07T08:01:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a2c00700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c15a610 con 0x7f389c0c9ec0 2023-10-07T08:01:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a099c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f389c0c9af0 0x7f389c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a099c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f389c0c9af0 0x7f389c15d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45156/0 (socket says 172.21.15.7:45156) 2023-10-07T08:01:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a099c700 1 -- 172.21.15.7:0/4079670938 learned_addr learned my addr 172.21.15.7:0/4079670938 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a119d700 1 --2- 172.21.15.7:0/4079670938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 0x7f389c06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a099c700 1 -- 172.21.15.7:0/4079670938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 msgr2=0x7f389c06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a099c700 1 --2- 172.21.15.7:0/4079670938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 0x7f389c06e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.356+0000 7f38a099c700 1 -- 172.21.15.7:0/4079670938 >> v1:172.21.15.172:6789/0 conn(0x7f389c063230 legacy=0x7f389c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.357+0000 7f38a099c700 1 -- 172.21.15.7:0/4079670938 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c15a8f0 con 0x7f389c0c9af0 2023-10-07T08:01:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.357+0000 7f38a099c700 1 --2- 172.21.15.7:0/4079670938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 0x7f389c15d8f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f38900099b0 tx=0x7f3890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588435bbbdb81dbd server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.357+0000 7f389b7fe700 1 -- 172.21.15.7:0/4079670938 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890009ea0 con 0x7f389c0c9af0 2023-10-07T08:01:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.357+0000 7f389b7fe700 1 -- 172.21.15.7:0/4079670938 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389000b860 con 0x7f389c0c9af0 2023-10-07T08:01:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.357+0000 7f389b7fe700 1 -- 172.21.15.7:0/4079670938 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389000bc00 con 0x7f389c0c9af0 2023-10-07T08:01:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.357+0000 7f38a2c00700 1 -- 172.21.15.7:0/4079670938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 msgr2=0x7f389c15d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.357+0000 7f38a2c00700 1 --2- 172.21.15.7:0/4079670938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 0x7f389c15d8f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f38900099b0 tx=0x7f3890009250 comp rx=0 tx=0).stop 2023-10-07T08:01:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.358+0000 7f38a2c00700 1 -- 172.21.15.7:0/4079670938 shutdown_connections 2023-10-07T08:01:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.358+0000 7f38a2c00700 1 --2- 172.21.15.7:0/4079670938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 0x7f389c06e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.358+0000 7f38a2c00700 1 --2- 172.21.15.7:0/4079670938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 0x7f389c15d8f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.358+0000 7f38a2c00700 1 -- 172.21.15.7:0/4079670938 >> 172.21.15.7:0/4079670938 conn(0x7f389c0cd1a0 msgr2=0x7f389c0c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.358+0000 7f38a2c00700 1 -- 172.21.15.7:0/4079670938 shutdown_connections 2023-10-07T08:01:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.358+0000 7f38a2c00700 1 -- 172.21.15.7:0/4079670938 wait complete. 2023-10-07T08:01:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.359+0000 7f38a2c00700 1 Processor -- start 2023-10-07T08:01:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.359+0000 7f38a2c00700 1 -- start start 2023-10-07T08:01:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f38a2c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f389c063230 0x7f389c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f38a2c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 0x7f389c155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f38a2c00700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 0x7f389c0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f38a2c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c155a60 con 0x7f389c0c9af0 2023-10-07T08:01:05.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f38a2c00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c156430 con 0x7f389c063230 2023-10-07T08:01:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f38a2c00700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c154e50 con 0x7f389c0c9ec0 2023-10-07T08:01:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f389bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 0x7f389c155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f389bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 0x7f389c155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45170/0 (socket says 172.21.15.7:45170) 2023-10-07T08:01:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f389bfff700 1 -- 172.21.15.7:0/986988718 learned_addr learned my addr 172.21.15.7:0/986988718 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f38a119d700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 0x7f389c0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f38a099c700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f389c063230 0x7f389c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f389bfff700 1 -- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 msgr2=0x7f389c0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f389bfff700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 0x7f389c0c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f389bfff700 1 -- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f389c063230 msgr2=0x7f389c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f389bfff700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f389c063230 0x7f389c154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.360+0000 7f389bfff700 1 -- 172.21.15.7:0/986988718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900094f0 con 0x7f389c0c9af0 2023-10-07T08:01:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.361+0000 7f389bfff700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 0x7f389c155ef0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f388800c590 tx=0x7f388800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.361+0000 7f3898ff9700 1 -- 172.21.15.7:0/986988718 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888011590 con 0x7f389c0c9af0 2023-10-07T08:01:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.361+0000 7f3898ff9700 1 -- 172.21.15.7:0/986988718 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3888011710 con 0x7f389c0c9af0 2023-10-07T08:01:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.361+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c155130 con 0x7f389c0c9af0 2023-10-07T08:01:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.361+0000 7f3898ff9700 1 -- 172.21.15.7:0/986988718 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888011ad0 con 0x7f389c0c9af0 2023-10-07T08:01:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.361+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c1556f0 con 0x7f389c0c9af0 2023-10-07T08:01:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.362+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c155bd0 con 0x7f389c0c9af0 2023-10-07T08:01:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.362+0000 7f3898ff9700 1 -- 172.21.15.7:0/986988718 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f388801b3b0 con 0x7f389c0c9af0 2023-10-07T08:01:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.363+0000 7f3898ff9700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3880066f60 0x7f3880069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.363+0000 7f3898ff9700 1 -- 172.21.15.7:0/986988718 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f38880836c0 con 0x7f389c0c9af0 2023-10-07T08:01:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.363+0000 7f38a099c700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3880066f60 0x7f3880069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.363+0000 7f38a099c700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3880066f60 0x7f3880069410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f38900099b0 tx=0x7f3890017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.367+0000 7f3898ff9700 1 -- 172.21.15.7:0/986988718 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3888050c30 con 0x7f389c0c9af0 2023-10-07T08:01:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.570+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f389c0716c0 con 0x7f3880066f60 2023-10-07T08:01:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.570+0000 7f3898ff9700 1 -- 172.21.15.7:0/986988718 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f389c0716c0 con 0x7f3880066f60 2023-10-07T08:01:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3880066f60 msgr2=0x7f3880069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3880066f60 0x7f3880069410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f38900099b0 tx=0x7f3890017000 comp rx=0 tx=0).stop 2023-10-07T08:01:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 msgr2=0x7f389c155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 0x7f389c155ef0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f388800c590 tx=0x7f388800c8a0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 shutdown_connections 2023-10-07T08:01:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f3880066f60 0x7f3880069410 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f389c0c9ec0 0x7f389c0c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f389c063230 0x7f389c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 --2- 172.21.15.7:0/986988718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f389c0c9af0 0x7f389c155ef0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 >> 172.21.15.7:0/986988718 conn(0x7f389c0cd1a0 msgr2=0x7f389c1502d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 shutdown_connections 2023-10-07T08:01:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.573+0000 7f38a2c00700 1 -- 172.21.15.7:0/986988718 wait complete. 2023-10-07T08:01:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2023-10-07T08:01:05.587 INFO:tasks.ceph:Scrubbing osd.3 2023-10-07T08:01:05.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-07T08:01:05.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.723+0000 7f4514ba7700 1 Processor -- start 2023-10-07T08:01:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.723+0000 7f4514ba7700 1 -- start start 2023-10-07T08:01:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.724+0000 7f4514ba7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 0x7f451015a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.724+0000 7f4514ba7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f451013c580 0x7f45100c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.724+0000 7f4514ba7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451015dac0 con 0x7f451006b4f0 2023-10-07T08:01:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.724+0000 7f4514ba7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451015df50 con 0x7f451013c580 2023-10-07T08:01:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.724+0000 7f4514ba7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451015d190 con 0x7f451013c1b0 2023-10-07T08:01:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450e59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 0x7f451015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450dd9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f451013c580 0x7f45100c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450e59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 0x7f451015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.7:36984/0 (socket says 172.21.15.7:36984) 2023-10-07T08:01:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450e59c700 1 -- 172.21.15.7:0/2440227292 learned_addr learned my addr 172.21.15.7:0/2440227292 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450dd9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f451013c580 0x7f45100c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:53066/0 (socket says 172.21.15.7:53066) 2023-10-07T08:01:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450dd9b700 1 -- 172.21.15.7:0/2440227292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 msgr2=0x7f451015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450dd9b700 1 --2- 172.21.15.7:0/2440227292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 0x7f451015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450dd9b700 1 -- 172.21.15.7:0/2440227292 >> v1:172.21.15.172:6789/0 conn(0x7f451006b4f0 legacy=0x7f451006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450d59a700 1 -- 172.21.15.7:0/2440227292 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294432715 0 0) 0x7f451015dac0 con 0x7f451006b4f0 2023-10-07T08:01:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450dd9b700 1 -- 172.21.15.7:0/2440227292 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451015d470 con 0x7f451013c580 2023-10-07T08:01:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450e59c700 1 --2- 172.21.15.7:0/2440227292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 0x7f451015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.725+0000 7f450dd9b700 1 --2- 172.21.15.7:0/2440227292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f451013c580 0x7f45100c2e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f45000098e0 tx=0x7f45000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80da8dc915bd26d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f450d59a700 1 -- 172.21.15.7:0/2440227292 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000b640 con 0x7f451013c580 2023-10-07T08:01:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f450d59a700 1 -- 172.21.15.7:0/2440227292 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450000b7a0 con 0x7f451013c580 2023-10-07T08:01:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f450d59a700 1 -- 172.21.15.7:0/2440227292 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000bb40 con 0x7f451013c580 2023-10-07T08:01:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f4514ba7700 1 -- 172.21.15.7:0/2440227292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f451013c580 msgr2=0x7f45100c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f4514ba7700 1 --2- 172.21.15.7:0/2440227292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f451013c580 0x7f45100c2e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f45000098e0 tx=0x7f45000091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f4514ba7700 1 -- 172.21.15.7:0/2440227292 shutdown_connections 2023-10-07T08:01:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f4514ba7700 1 --2- 172.21.15.7:0/2440227292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 0x7f451015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f4514ba7700 1 --2- 172.21.15.7:0/2440227292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f451013c580 0x7f45100c2e20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f4514ba7700 1 -- 172.21.15.7:0/2440227292 >> 172.21.15.7:0/2440227292 conn(0x7f4510067a70 msgr2=0x7f45100ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f4514ba7700 1 -- 172.21.15.7:0/2440227292 shutdown_connections 2023-10-07T08:01:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.726+0000 7f4514ba7700 1 -- 172.21.15.7:0/2440227292 wait complete. 2023-10-07T08:01:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.728+0000 7f4514ba7700 1 Processor -- start 2023-10-07T08:01:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.728+0000 7f4514ba7700 1 -- start start 2023-10-07T08:01:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.728+0000 7f4514ba7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451006b4f0 0x7f4510152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.728+0000 7f4514ba7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 0x7f4510169b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.728+0000 7f4514ba7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f451013c580 0x7f45100b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.728+0000 7f4514ba7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45100b1ee0 con 0x7f451006b4f0 2023-10-07T08:01:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.728+0000 7f4514ba7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45100b34c0 con 0x7f451013c580 2023-10-07T08:01:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f4514ba7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451013ca80 con 0x7f451013c1b0 2023-10-07T08:01:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451006b4f0 0x7f4510152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451006b4f0 0x7f4510152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45176/0 (socket says 172.21.15.7:45176) 2023-10-07T08:01:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450e59c700 1 -- 172.21.15.7:0/1957509035 learned_addr learned my addr 172.21.15.7:0/1957509035 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450dd9b700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 0x7f4510169b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450ed9d700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f451013c580 0x7f45100b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450e59c700 1 -- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 msgr2=0x7f4510169b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450e59c700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f451013c1b0 0x7f4510169b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450e59c700 1 -- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f451013c580 msgr2=0x7f45100b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450e59c700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f451013c580 0x7f45100b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450e59c700 1 -- 172.21.15.7:0/1957509035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040094f0 con 0x7f451006b4f0 2023-10-07T08:01:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f450e59c700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451006b4f0 0x7f4510152fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f450400ec50 tx=0x7f4504009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f44fe7fc700 1 -- 172.21.15.7:0/1957509035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45040097d0 con 0x7f451006b4f0 2023-10-07T08:01:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4500009420 con 0x7f451006b4f0 2023-10-07T08:01:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f44fe7fc700 1 -- 172.21.15.7:0/1957509035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450400c9d0 con 0x7f451006b4f0 2023-10-07T08:01:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f44fe7fc700 1 -- 172.21.15.7:0/1957509035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450400cd50 con 0x7f451006b4f0 2023-10-07T08:01:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.729+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451013d040 con 0x7f451006b4f0 2023-10-07T08:01:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.730+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f45100716c0 con 0x7f451006b4f0 2023-10-07T08:01:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.731+0000 7f44fe7fc700 1 -- 172.21.15.7:0/1957509035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4504013680 con 0x7f451006b4f0 2023-10-07T08:01:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.731+0000 7f44fe7fc700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f44f0066f30 0x7f44f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.731+0000 7f44fe7fc700 1 -- 172.21.15.7:0/1957509035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f45040849a0 con 0x7f451006b4f0 2023-10-07T08:01:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.731+0000 7f44fe7fc700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7f44f006cd80 0x7f44f006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.731+0000 7f44fe7fc700 1 -- 172.21.15.7:0/1957509035 --> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44f006f860 con 0x7f44f006cd80 2023-10-07T08:01:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.731+0000 7f44fe7fc700 1 -- 172.21.15.7:0/1957509035 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4504089240 con 0x7f451006b4f0 2023-10-07T08:01:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.732+0000 7f450dd9b700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f44f0066f30 0x7f44f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.732+0000 7f450ed9d700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7f44f006cd80 0x7f44f006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.732+0000 7f450ed9d700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7f44f006cd80 0x7f44f006f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.732+0000 7f450dd9b700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f44f0066f30 0x7f44f00693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f45000098e0 tx=0x7f4500017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.733+0000 7f44fe7fc700 1 -- 172.21.15.7:0/1957509035 <== osd.3 v2:172.21.15.7:6800/120001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f44f006f860 con 0x7f44f006cd80 2023-10-07T08:01:05.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.747+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 --> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f45100ce2b0 con 0x7f44f006cd80 2023-10-07T08:01:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f44fe7fc700 1 -- 172.21.15.7:0/1957509035 <== osd.3 v2:172.21.15.7:6800/120001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f45100ce2b0 con 0x7f44f006cd80 2023-10-07T08:01:05.749 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T08:01:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T08:01:05.749 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T08:01:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7f44f006cd80 msgr2=0x7f44f006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.7:6800/120001,v1:172.21.15.7:6801/120001] conn(0x7f44f006cd80 0x7f44f006f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f44f0066f30 msgr2=0x7f44f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f44f0066f30 0x7f44f00693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f45000098e0 tx=0x7f4500017000 comp rx=0 tx=0).stop 2023-10-07T08:01:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451006b4f0 msgr2=0x7f4510152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 --2- 172.21.15.7:0/1957509035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451006b4f0 0x7f4510152fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f450400ec50 tx=0x7f4504009250 comp rx=0 tx=0).stop 2023-10-07T08:01:05.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f450ed9d700 1 -- 172.21.15.7:0/1957509035 reap_dead start 2023-10-07T08:01:05.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 shutdown_connections 2023-10-07T08:01:05.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 >> 172.21.15.7:0/1957509035 conn(0x7f4510067a70 msgr2=0x7f4510069400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:05.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 shutdown_connections 2023-10-07T08:01:05.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.748+0000 7f4514ba7700 1 -- 172.21.15.7:0/1957509035 wait complete. 2023-10-07T08:01:05.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-07T08:01:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.895+0000 7f674235d700 1 Processor -- start 2023-10-07T08:01:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.895+0000 7f674235d700 1 -- start start 2023-10-07T08:01:05.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f674235d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f673c159310 0x7f673c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f674235d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c1596e0 0x7f673c06b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f674235d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673c068100 con 0x7f673c0ced30 2023-10-07T08:01:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f674235d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c068350 con 0x7f673c159310 2023-10-07T08:01:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f674235d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c068630 con 0x7f673c1596e0 2023-10-07T08:01:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f673bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f673c159310 0x7f673c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f673b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f673c0ced30 0x7f673c0c3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47050/0 (socket says 172.21.15.7:47050) 2023-10-07T08:01:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f673b7fe700 1 -- 172.21.15.7:0/3928206477 learned_addr learned my addr 172.21.15.7:0/3928206477 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f673bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f673c159310 0x7f673c0c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45184/0 (socket says 172.21.15.7:45184) 2023-10-07T08:01:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f67408fa700 1 --2- 172.21.15.7:0/3928206477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c1596e0 0x7f673c06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f673bfff700 1 -- 172.21.15.7:0/3928206477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c1596e0 msgr2=0x7f673c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f673bfff700 1 --2- 172.21.15.7:0/3928206477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c1596e0 0x7f673c06b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f673bfff700 1 -- 172.21.15.7:0/3928206477 >> v1:172.21.15.7:6789/0 conn(0x7f673c0ced30 legacy=0x7f673c0c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f673bfff700 1 -- 172.21.15.7:0/3928206477 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c068910 con 0x7f673c159310 2023-10-07T08:01:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.896+0000 7f673affd700 1 -- 172.21.15.7:0/3928206477 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069326985 0 0) 0x7f673c068100 con 0x7f673c0ced30 2023-10-07T08:01:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f673bfff700 1 --2- 172.21.15.7:0/3928206477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c159310 0x7f673c0c86e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f672c0098e0 tx=0x7f672c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37ea5c87174eee3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f673affd700 1 -- 172.21.15.7:0/3928206477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00b640 con 0x7f673c159310 2023-10-07T08:01:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f673affd700 1 -- 172.21.15.7:0/3928206477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c00b7a0 con 0x7f673c159310 2023-10-07T08:01:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f673affd700 1 -- 172.21.15.7:0/3928206477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00bb40 con 0x7f673c159310 2023-10-07T08:01:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f674235d700 1 -- 172.21.15.7:0/3928206477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c159310 msgr2=0x7f673c0c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f674235d700 1 --2- 172.21.15.7:0/3928206477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c159310 0x7f673c0c86e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f672c0098e0 tx=0x7f672c0091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f674235d700 1 -- 172.21.15.7:0/3928206477 shutdown_connections 2023-10-07T08:01:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f674235d700 1 --2- 172.21.15.7:0/3928206477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c1596e0 0x7f673c06b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f674235d700 1 --2- 172.21.15.7:0/3928206477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c159310 0x7f673c0c86e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f674235d700 1 -- 172.21.15.7:0/3928206477 >> 172.21.15.7:0/3928206477 conn(0x7f673c067ac0 msgr2=0x7f673c0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.897+0000 7f674235d700 1 -- 172.21.15.7:0/3928206477 shutdown_connections 2023-10-07T08:01:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.898+0000 7f674235d700 1 -- 172.21.15.7:0/3928206477 wait complete. 2023-10-07T08:01:05.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.899+0000 7f674235d700 1 Processor -- start 2023-10-07T08:01:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.899+0000 7f674235d700 1 -- start start 2023-10-07T08:01:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f674235d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c159310 0x7f673c0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f674235d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c1596e0 0x7f673c154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f674235d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c154c80 0x7f673c16ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f674235d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c16db50 con 0x7f673c1596e0 2023-10-07T08:01:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f674235d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0c5c40 con 0x7f673c159310 2023-10-07T08:01:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f674235d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c155120 con 0x7f673c154c80 2023-10-07T08:01:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c1596e0 0x7f673c154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c1596e0 0x7f673c154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45198/0 (socket says 172.21.15.7:45198) 2023-10-07T08:01:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673b7fe700 1 -- 172.21.15.7:0/3511001582 learned_addr learned my addr 172.21.15.7:0/3511001582 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f67408fa700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c154c80 0x7f673c16ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673bfff700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c159310 0x7f673c0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673b7fe700 1 -- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c154c80 msgr2=0x7f673c16ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673b7fe700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c154c80 0x7f673c16ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673b7fe700 1 -- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c159310 msgr2=0x7f673c0c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673b7fe700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c159310 0x7f673c0c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673b7fe700 1 -- 172.21.15.7:0/3511001582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c009420 con 0x7f673c1596e0 2023-10-07T08:01:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f673b7fe700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c1596e0 0x7f673c154740 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f672400c810 tx=0x7f6724007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f6733fff700 1 -- 172.21.15.7:0/3511001582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724005b30 con 0x7f673c1596e0 2023-10-07T08:01:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.900+0000 7f6733fff700 1 -- 172.21.15.7:0/3511001582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6724005c90 con 0x7f673c1596e0 2023-10-07T08:01:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.901+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c16a130 con 0x7f673c1596e0 2023-10-07T08:01:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.901+0000 7f6733fff700 1 -- 172.21.15.7:0/3511001582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724007300 con 0x7f673c1596e0 2023-10-07T08:01:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.901+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c169ae0 con 0x7f673c1596e0 2023-10-07T08:01:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.902+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c16a2c0 con 0x7f673c1596e0 2023-10-07T08:01:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.902+0000 7f6733fff700 1 -- 172.21.15.7:0/3511001582 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67240076b0 con 0x7f673c1596e0 2023-10-07T08:01:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.903+0000 7f6733fff700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f671c066f60 0x7f671c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.903+0000 7f6733fff700 1 -- 172.21.15.7:0/3511001582 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6724083680 con 0x7f673c1596e0 2023-10-07T08:01:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.903+0000 7f673bfff700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f671c066f60 0x7f671c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.903+0000 7f673bfff700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f671c066f60 0x7f671c069410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f672c0093f0 tx=0x7f672c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:05.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:05.907+0000 7f6733fff700 1 -- 172.21.15.7:0/3511001582 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6724050bf0 con 0x7f673c1596e0 2023-10-07T08:01:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.108+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f673c1695f0 con 0x7f671c066f60 2023-10-07T08:01:06.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.109+0000 7f6733fff700 1 -- 172.21.15.7:0/3511001582 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f673c1695f0 con 0x7f671c066f60 2023-10-07T08:01:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f671c066f60 msgr2=0x7f671c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f671c066f60 0x7f671c069410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f672c0093f0 tx=0x7f672c017000 comp rx=0 tx=0).stop 2023-10-07T08:01:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c1596e0 msgr2=0x7f673c154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c1596e0 0x7f673c154740 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f672400c810 tx=0x7f6724007df0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 shutdown_connections 2023-10-07T08:01:06.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f671c066f60 0x7f671c069410 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673c154c80 0x7f673c16ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c159310 0x7f673c0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 --2- 172.21.15.7:0/3511001582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f673c1596e0 0x7f673c154740 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 >> 172.21.15.7:0/3511001582 conn(0x7f673c067ac0 msgr2=0x7f673c0cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 shutdown_connections 2023-10-07T08:01:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.111+0000 7f674235d700 1 -- 172.21.15.7:0/3511001582 wait complete. 2023-10-07T08:01:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2023-10-07T08:01:06.126 INFO:tasks.ceph:Scrubbing osd.4 2023-10-07T08:01:06.127 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-07T08:01:06.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.262+0000 7f18986b2700 1 Processor -- start 2023-10-07T08:01:06.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.262+0000 7f18986b2700 1 -- start start 2023-10-07T08:01:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.263+0000 7f18986b2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18901598c0 0x7f189006af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.263+0000 7f18986b2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18900a9920 con 0x7f18900c1d30 2023-10-07T08:01:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.263+0000 7f18986b2700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189006b4f0 con 0x7f18901594f0 2023-10-07T08:01:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.263+0000 7f18986b2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189006b7d0 con 0x7f18901598c0 2023-10-07T08:01:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.263+0000 7f1896c4f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f18900c1d30 0x7f18900a9db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47066/0 (socket says 172.21.15.7:47066) 2023-10-07T08:01:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.263+0000 7f1896c4f700 1 -- 172.21.15.7:0/158053053 learned_addr learned my addr 172.21.15.7:0/158053053 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.263+0000 7f1895c4d700 1 --2- 172.21.15.7:0/158053053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18901598c0 0x7f189006af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f189544c700 1 -- 172.21.15.7:0/158053053 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786574968 0 0) 0x7f18900a9920 con 0x7f18900c1d30 2023-10-07T08:01:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f189544c700 1 -- 172.21.15.7:0/158053053 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f187c003570 con 0x7f18900c1d30 2023-10-07T08:01:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f189544c700 1 -- 172.21.15.7:0/158053053 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110330251 0 0) 0x7f189006b4f0 con 0x7f18901594f0 2023-10-07T08:01:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f189544c700 1 -- 172.21.15.7:0/158053053 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18900a9920 con 0x7f18901594f0 2023-10-07T08:01:06.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f1895c4d700 1 -- 172.21.15.7:0/158053053 >> v1:172.21.15.172:6789/0 conn(0x7f18901594f0 legacy=0x7f18900674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f1895c4d700 1 -- 172.21.15.7:0/158053053 >> v1:172.21.15.7:6789/0 conn(0x7f18900c1d30 legacy=0x7f18900a9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f1895c4d700 1 -- 172.21.15.7:0/158053053 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189006bab0 con 0x7f18901598c0 2023-10-07T08:01:06.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f1895c4d700 1 --2- 172.21.15.7:0/158053053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18901598c0 0x7f189006af10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1880009910 tx=0x7f18800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe20685c8ac998b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f189544c700 1 -- 172.21.15.7:0/158053053 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880011630 con 0x7f18901598c0 2023-10-07T08:01:06.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f189544c700 1 -- 172.21.15.7:0/158053053 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1880011790 con 0x7f18901598c0 2023-10-07T08:01:06.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.264+0000 7f189544c700 1 -- 172.21.15.7:0/158053053 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880011b30 con 0x7f18901598c0 2023-10-07T08:01:06.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.265+0000 7f18986b2700 1 -- 172.21.15.7:0/158053053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18901598c0 msgr2=0x7f189006af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.265+0000 7f18986b2700 1 --2- 172.21.15.7:0/158053053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18901598c0 0x7f189006af10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1880009910 tx=0x7f18800091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.265+0000 7f18986b2700 1 -- 172.21.15.7:0/158053053 shutdown_connections 2023-10-07T08:01:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.265+0000 7f18986b2700 1 --2- 172.21.15.7:0/158053053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18901598c0 0x7f189006af10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.265+0000 7f18986b2700 1 -- 172.21.15.7:0/158053053 >> 172.21.15.7:0/158053053 conn(0x7f18900cda60 msgr2=0x7f1890068100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.265+0000 7f18986b2700 1 -- 172.21.15.7:0/158053053 shutdown_connections 2023-10-07T08:01:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.265+0000 7f18986b2700 1 -- 172.21.15.7:0/158053053 wait complete. 2023-10-07T08:01:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.266+0000 7f18986b2700 1 Processor -- start 2023-10-07T08:01:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.266+0000 7f18986b2700 1 -- start start 2023-10-07T08:01:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f18986b2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18900c1d30 0x7f1890154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f18986b2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18901594f0 0x7f189015d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f18986b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18901598c0 0x7f189016cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f18986b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189015e720 con 0x7f18901598c0 2023-10-07T08:01:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f18986b2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1890155210 con 0x7f18901594f0 2023-10-07T08:01:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f18986b2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189015db10 con 0x7f18900c1d30 2023-10-07T08:01:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f1896c4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18901598c0 0x7f189016cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f1896c4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18901598c0 0x7f189016cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45210/0 (socket says 172.21.15.7:45210) 2023-10-07T08:01:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f1896c4f700 1 -- 172.21.15.7:0/2605122031 learned_addr learned my addr 172.21.15.7:0/2605122031 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f189644e700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18900c1d30 0x7f1890154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.267+0000 7f1895c4d700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18901594f0 0x7f189015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f1896c4f700 1 -- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18900c1d30 msgr2=0x7f1890154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f1896c4f700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18900c1d30 0x7f1890154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f1896c4f700 1 -- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18901594f0 msgr2=0x7f189015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f1896c4f700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18901594f0 0x7f189015d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f1896c4f700 1 -- 172.21.15.7:0/2605122031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1880009420 con 0x7f18901598c0 2023-10-07T08:01:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f1896c4f700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18901598c0 0x7f189016cf50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f188800cc60 tx=0x7f1888001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f18867fc700 1 -- 172.21.15.7:0/2605122031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188800a8b0 con 0x7f18901598c0 2023-10-07T08:01:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f18867fc700 1 -- 172.21.15.7:0/2605122031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188800aa30 con 0x7f18901598c0 2023-10-07T08:01:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f18867fc700 1 -- 172.21.15.7:0/2605122031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188800adf0 con 0x7f18901598c0 2023-10-07T08:01:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189015ddf0 con 0x7f18901598c0 2023-10-07T08:01:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.268+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189015e3b0 con 0x7f18901598c0 2023-10-07T08:01:06.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.269+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18900716c0 con 0x7f18901598c0 2023-10-07T08:01:06.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.270+0000 7f18867fc700 1 -- 172.21.15.7:0/2605122031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1888007300 con 0x7f18901598c0 2023-10-07T08:01:06.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.270+0000 7f18867fc700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1874066f30 0x7f18740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.270+0000 7f18867fc700 1 -- 172.21.15.7:0/2605122031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1888084560 con 0x7f18901598c0 2023-10-07T08:01:06.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.270+0000 7f18867fc700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7f187406cd80 0x7f187406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.270+0000 7f18867fc700 1 -- 172.21.15.7:0/2605122031 --> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f187406f860 con 0x7f187406cd80 2023-10-07T08:01:06.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.270+0000 7f18867fc700 1 -- 172.21.15.7:0/2605122031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1888086b40 con 0x7f18901598c0 2023-10-07T08:01:06.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.270+0000 7f189644e700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1874066f30 0x7f18740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.270+0000 7f1895c4d700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7f187406cd80 0x7f187406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.271+0000 7f189644e700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1874066f30 0x7f18740693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f188c007fd0 tx=0x7f188c007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.271+0000 7f1895c4d700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7f187406cd80 0x7f187406f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.272+0000 7f18867fc700 1 -- 172.21.15.7:0/2605122031 <== osd.4 v2:172.21.15.172:6800/120243 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f187406f860 con 0x7f187406cd80 2023-10-07T08:01:06.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.286+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 --> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f18900694a0 con 0x7f187406cd80 2023-10-07T08:01:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.287+0000 7f18867fc700 1 -- 172.21.15.7:0/2605122031 <== osd.4 v2:172.21.15.172:6800/120243 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f18900694a0 con 0x7f187406cd80 2023-10-07T08:01:06.289 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T08:01:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T08:01:06.289 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T08:01:06.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7f187406cd80 msgr2=0x7f187406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6800/120243,v1:172.21.15.172:6801/120243] conn(0x7f187406cd80 0x7f187406f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1874066f30 msgr2=0x7f18740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f1874066f30 0x7f18740693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f188c007fd0 tx=0x7f188c007d50 comp rx=0 tx=0).stop 2023-10-07T08:01:06.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18901598c0 msgr2=0x7f189016cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 --2- 172.21.15.7:0/2605122031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18901598c0 0x7f189016cf50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f188800cc60 tx=0x7f1888001ae0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f1896c4f700 1 -- 172.21.15.7:0/2605122031 reap_dead start 2023-10-07T08:01:06.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 shutdown_connections 2023-10-07T08:01:06.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 >> 172.21.15.7:0/2605122031 conn(0x7f18900cda60 msgr2=0x7f189006f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:06.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 shutdown_connections 2023-10-07T08:01:06.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.288+0000 7f18986b2700 1 -- 172.21.15.7:0/2605122031 wait complete. 2023-10-07T08:01:06.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-07T08:01:06.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.435+0000 7f26fe4c2700 1 Processor -- start 2023-10-07T08:01:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.435+0000 7f26fe4c2700 1 -- start start 2023-10-07T08:01:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26fe4c2700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26f8155f20 0x7f26f80c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26fe4c2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26fe4c2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f806b700 con 0x7f26f806e2c0 2023-10-07T08:01:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26fe4c2700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f806aaf0 con 0x7f26f8155f20 2023-10-07T08:01:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26fe4c2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f806add0 con 0x7f26f81562f0 2023-10-07T08:01:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26fca5f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f26f806e2c0 0x7f26f8159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47070/0 (socket says 172.21.15.7:47070) 2023-10-07T08:01:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26fca5f700 1 -- 172.21.15.7:0/1935395907 learned_addr learned my addr 172.21.15.7:0/1935395907 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f7fff700 1 --2- 172.21.15.7:0/1935395907 >> v2:172.21.15.172:3300/0 conn(0x7f26f8155f20 0x7f26f80c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f77fe700 1 --2- 172.21.15.7:0/1935395907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f6ffd700 1 -- 172.21.15.7:0/1935395907 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709015222 0 0) 0x7f26f806b700 con 0x7f26f806e2c0 2023-10-07T08:01:06.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f6ffd700 1 -- 172.21.15.7:0/1935395907 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e0003570 con 0x7f26f806e2c0 2023-10-07T08:01:06.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f6ffd700 1 -- 172.21.15.7:0/1935395907 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 154536419 0 0) 0x7f26e0003570 con 0x7f26f806e2c0 2023-10-07T08:01:06.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f77fe700 1 -- 172.21.15.7:0/1935395907 >> v2:172.21.15.172:3300/0 conn(0x7f26f8155f20 msgr2=0x7f26f80c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:06.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f77fe700 1 --2- 172.21.15.7:0/1935395907 >> v2:172.21.15.172:3300/0 conn(0x7f26f8155f20 0x7f26f80c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f77fe700 1 -- 172.21.15.7:0/1935395907 >> v1:172.21.15.7:6789/0 conn(0x7f26f806e2c0 legacy=0x7f26f8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f77fe700 1 -- 172.21.15.7:0/1935395907 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f806b0b0 con 0x7f26f81562f0 2023-10-07T08:01:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.437+0000 7f26f7fff700 1 --2- 172.21.15.7:0/1935395907 >> v2:172.21.15.172:3300/0 conn(0x7f26f8155f20 0x7f26f80c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26f77fe700 1 --2- 172.21.15.7:0/1935395907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f806a3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0098e0 tx=0x7f26ec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60ac6314a889ccd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26f6ffd700 1 -- 172.21.15.7:0/1935395907 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00017f0 con 0x7f26f81562f0 2023-10-07T08:01:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26f6ffd700 1 -- 172.21.15.7:0/1935395907 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec011420 con 0x7f26f81562f0 2023-10-07T08:01:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26f6ffd700 1 -- 172.21.15.7:0/1935395907 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0117c0 con 0x7f26f81562f0 2023-10-07T08:01:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26fe4c2700 1 -- 172.21.15.7:0/1935395907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 msgr2=0x7f26f806a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26fe4c2700 1 --2- 172.21.15.7:0/1935395907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f806a3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0098e0 tx=0x7f26ec0091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26fe4c2700 1 -- 172.21.15.7:0/1935395907 shutdown_connections 2023-10-07T08:01:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26fe4c2700 1 --2- 172.21.15.7:0/1935395907 >> v2:172.21.15.172:3300/0 conn(0x7f26f8155f20 0x7f26f80c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26fe4c2700 1 --2- 172.21.15.7:0/1935395907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f806a3f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.438+0000 7f26fe4c2700 1 -- 172.21.15.7:0/1935395907 >> 172.21.15.7:0/1935395907 conn(0x7f26f8068bd0 msgr2=0x7f26f80ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.439+0000 7f26fe4c2700 1 -- 172.21.15.7:0/1935395907 shutdown_connections 2023-10-07T08:01:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.439+0000 7f26fe4c2700 1 -- 172.21.15.7:0/1935395907 wait complete. 2023-10-07T08:01:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.440+0000 7f26fe4c2700 1 Processor -- start 2023-10-07T08:01:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.440+0000 7f26fe4c2700 1 -- start start 2023-10-07T08:01:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.441+0000 7f26fe4c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f806e2c0 0x7f26f80c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.441+0000 7f26fe4c2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26f8155f20 0x7f26f80c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.441+0000 7f26fe4c2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.441+0000 7f26fe4c2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80c5930 con 0x7f26f806e2c0 2023-10-07T08:01:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.441+0000 7f26fe4c2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80c5c10 con 0x7f26f8155f20 2023-10-07T08:01:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.441+0000 7f26fe4c2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80c5ef0 con 0x7f26f81562f0 2023-10-07T08:01:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f806e2c0 0x7f26f80c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f806e2c0 0x7f26f80c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45216/0 (socket says 172.21.15.7:45216) 2023-10-07T08:01:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f7fff700 1 -- 172.21.15.7:0/799879513 learned_addr learned my addr 172.21.15.7:0/799879513 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f77fe700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26f8155f20 0x7f26f80c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26fca5f700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f7fff700 1 -- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 msgr2=0x7f26f814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f7fff700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f814f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f7fff700 1 -- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26f8155f20 msgr2=0x7f26f80c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f7fff700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26f8155f20 0x7f26f80c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f7fff700 1 -- 172.21.15.7:0/799879513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8009420 con 0x7f26f806e2c0 2023-10-07T08:01:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f77fe700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26f8155f20 0x7f26f80c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:01:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26fca5f700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:01:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26f7fff700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f806e2c0 0x7f26f80c5030 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f26e800ec80 tx=0x7f26e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26dffff700 1 -- 172.21.15.7:0/799879513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800c9d0 con 0x7f26f806e2c0 2023-10-07T08:01:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26dffff700 1 -- 172.21.15.7:0/799879513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e800cb30 con 0x7f26f806e2c0 2023-10-07T08:01:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26dffff700 1 -- 172.21.15.7:0/799879513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800ceb0 con 0x7f26f806e2c0 2023-10-07T08:01:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec009420 con 0x7f26f806e2c0 2023-10-07T08:01:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.442+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f8158950 con 0x7f26f806e2c0 2023-10-07T08:01:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.444+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f80716c0 con 0x7f26f806e2c0 2023-10-07T08:01:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.444+0000 7f26dffff700 1 -- 172.21.15.7:0/799879513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26e8013680 con 0x7f26f806e2c0 2023-10-07T08:01:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.444+0000 7f26dffff700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f26e4066f30 0x7f26e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.444+0000 7f26dffff700 1 -- 172.21.15.7:0/799879513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f26e80846b0 con 0x7f26f806e2c0 2023-10-07T08:01:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.445+0000 7f26f77fe700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f26e4066f30 0x7f26e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.445+0000 7f26f77fe700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f26e4066f30 0x7f26e40693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f26f806a8e0 tx=0x7f26ec016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.449+0000 7f26dffff700 1 -- 172.21.15.7:0/799879513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26e8051c20 con 0x7f26f806e2c0 2023-10-07T08:01:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.651+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f26f806f1f0 con 0x7f26e4066f30 2023-10-07T08:01:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.652+0000 7f26dffff700 1 -- 172.21.15.7:0/799879513 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f26f806f1f0 con 0x7f26e4066f30 2023-10-07T08:01:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f26e4066f30 msgr2=0x7f26e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f26e4066f30 0x7f26e40693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f26f806a8e0 tx=0x7f26ec016000 comp rx=0 tx=0).stop 2023-10-07T08:01:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f806e2c0 msgr2=0x7f26f80c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f806e2c0 0x7f26f80c5030 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f26e800ec80 tx=0x7f26e80091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 shutdown_connections 2023-10-07T08:01:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f26e4066f30 0x7f26e40693e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26f81562f0 0x7f26f814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f806e2c0 0x7f26f80c5030 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 --2- 172.21.15.7:0/799879513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26f8155f20 0x7f26f80c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 >> 172.21.15.7:0/799879513 conn(0x7f26f8068bd0 msgr2=0x7f26f806b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 shutdown_connections 2023-10-07T08:01:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.654+0000 7f26fe4c2700 1 -- 172.21.15.7:0/799879513 wait complete. 2023-10-07T08:01:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2023-10-07T08:01:06.670 INFO:tasks.ceph:Scrubbing osd.5 2023-10-07T08:01:06.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-07T08:01:06.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.806+0000 7f289c288700 1 Processor -- start 2023-10-07T08:01:06.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.806+0000 7f289c288700 1 -- start start 2023-10-07T08:01:06.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f289c288700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28941585f0 0x7f289406b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f289c288700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2894151b20 0x7f289406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f289c288700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2894157890 con 0x7f2894151750 2023-10-07T08:01:06.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f289c288700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28940c8f40 con 0x7f2894151b20 2023-10-07T08:01:06.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f289c288700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940c8c60 con 0x7f28941585f0 2023-10-07T08:01:06.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f289a825700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2894151b20 0x7f289406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f2899823700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28941585f0 0x7f289406b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f289a825700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2894151b20 0x7f289406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:53122/0 (socket says 172.21.15.7:53122) 2023-10-07T08:01:06.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f289a825700 1 -- 172.21.15.7:0/2936169413 learned_addr learned my addr 172.21.15.7:0/2936169413 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:06.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.807+0000 7f289a024700 1 --1- 172.21.15.7:0/2936169413 >> v1:172.21.15.172:6789/0 conn(0x7f2894151750 0x7f289415b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:43478/0 (socket says 172.21.15.7:43478) 2023-10-07T08:01:06.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f289a825700 1 -- 172.21.15.7:0/2936169413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28941585f0 msgr2=0x7f289406b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:06.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f289a825700 1 --2- 172.21.15.7:0/2936169413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28941585f0 0x7f289406b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f289a825700 1 -- 172.21.15.7:0/2936169413 >> v1:172.21.15.172:6789/0 conn(0x7f2894151750 legacy=0x7f289415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f289a825700 1 -- 172.21.15.7:0/2936169413 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2894157b10 con 0x7f2894151b20 2023-10-07T08:01:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f2899022700 1 -- 172.21.15.7:0/2936169413 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041368134 0 0) 0x7f2894157890 con 0x7f2894151750 2023-10-07T08:01:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f2899823700 1 --2- 172.21.15.7:0/2936169413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28941585f0 0x7f289406b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f289a825700 1 --2- 172.21.15.7:0/2936169413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2894151b20 0x7f289406ba30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f288c009f20 tx=0x7f288c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a331e979b70b798 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f2899022700 1 -- 172.21.15.7:0/2936169413 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c00e070 con 0x7f2894151b20 2023-10-07T08:01:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f2899022700 1 -- 172.21.15.7:0/2936169413 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c00b9d0 con 0x7f2894151b20 2023-10-07T08:01:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.808+0000 7f2899022700 1 -- 172.21.15.7:0/2936169413 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c0092e0 con 0x7f2894151b20 2023-10-07T08:01:06.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.809+0000 7f289c288700 1 -- 172.21.15.7:0/2936169413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2894151b20 msgr2=0x7f289406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.809+0000 7f289c288700 1 --2- 172.21.15.7:0/2936169413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2894151b20 0x7f289406ba30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f288c009f20 tx=0x7f288c009840 comp rx=0 tx=0).stop 2023-10-07T08:01:06.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.809+0000 7f289c288700 1 -- 172.21.15.7:0/2936169413 shutdown_connections 2023-10-07T08:01:06.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.809+0000 7f289c288700 1 --2- 172.21.15.7:0/2936169413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2894151b20 0x7f289406ba30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.809+0000 7f289c288700 1 --2- 172.21.15.7:0/2936169413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28941585f0 0x7f289406b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.809+0000 7f289c288700 1 -- 172.21.15.7:0/2936169413 >> 172.21.15.7:0/2936169413 conn(0x7f2894067a70 msgr2=0x7f28940ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:06.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.809+0000 7f289c288700 1 -- 172.21.15.7:0/2936169413 shutdown_connections 2023-10-07T08:01:06.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.809+0000 7f289c288700 1 -- 172.21.15.7:0/2936169413 wait complete. 2023-10-07T08:01:06.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.810+0000 7f289c288700 1 Processor -- start 2023-10-07T08:01:06.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.810+0000 7f289c288700 1 -- start start 2023-10-07T08:01:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289c288700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894151750 0x7f28940b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289c288700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2894151b20 0x7f28940c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289c288700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28941585f0 0x7f28940b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289c288700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b2ab0 con 0x7f2894151750 2023-10-07T08:01:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289c288700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28941507d0 con 0x7f28941585f0 2023-10-07T08:01:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289c288700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b2700 con 0x7f2894151b20 2023-10-07T08:01:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a024700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894151750 0x7f28940b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a024700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894151750 0x7f28940b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45222/0 (socket says 172.21.15.7:45222) 2023-10-07T08:01:06.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a024700 1 -- 172.21.15.7:0/2447742302 learned_addr learned my addr 172.21.15.7:0/2447742302 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:06.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a825700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28941585f0 0x7f28940b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f2899823700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2894151b20 0x7f28940c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a825700 1 -- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2894151b20 msgr2=0x7f28940c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a825700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2894151b20 0x7f28940c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a825700 1 -- 172.21.15.7:0/2447742302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894151750 msgr2=0x7f28940b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a825700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894151750 0x7f28940b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a825700 1 -- 172.21.15.7:0/2447742302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884009420 con 0x7f28941585f0 2023-10-07T08:01:06.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a024700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894151750 0x7f28940b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:06.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.811+0000 7f289a825700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28941585f0 0x7f28940b3780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f288c015000 tx=0x7f288c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.812+0000 7f288a7fc700 1 -- 172.21.15.7:0/2447742302 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c008b10 con 0x7f28941585f0 2023-10-07T08:01:06.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.812+0000 7f288a7fc700 1 -- 172.21.15.7:0/2447742302 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c008cb0 con 0x7f28941585f0 2023-10-07T08:01:06.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.812+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c009ae0 con 0x7f28941585f0 2023-10-07T08:01:06.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.812+0000 7f288a7fc700 1 -- 172.21.15.7:0/2447742302 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c0266b0 con 0x7f28941585f0 2023-10-07T08:01:06.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.812+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28940b2fa0 con 0x7f28941585f0 2023-10-07T08:01:06.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.812+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28940716c0 con 0x7f28941585f0 2023-10-07T08:01:06.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.813+0000 7f288a7fc700 1 -- 172.21.15.7:0/2447742302 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f288c003040 con 0x7f28941585f0 2023-10-07T08:01:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.814+0000 7f288a7fc700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2878066f30 0x7f28780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.814+0000 7f288a7fc700 1 -- 172.21.15.7:0/2447742302 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f288c086630 con 0x7f28941585f0 2023-10-07T08:01:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.814+0000 7f288a7fc700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f287806cd80 0x7f287806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.814+0000 7f288a7fc700 1 -- 172.21.15.7:0/2447742302 --> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f287806f860 con 0x7f287806cd80 2023-10-07T08:01:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.814+0000 7f288a7fc700 1 -- 172.21.15.7:0/2447742302 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f288c086960 con 0x7f28941585f0 2023-10-07T08:01:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.814+0000 7f289a024700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2878066f30 0x7f28780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.814+0000 7f2899823700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f287806cd80 0x7f287806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.815+0000 7f289a024700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2878066f30 0x7f28780693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2890007fd0 tx=0x7f2890007e10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.815+0000 7f2899823700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f287806cd80 0x7f287806f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.816+0000 7f288a7fc700 1 -- 172.21.15.7:0/2447742302 <== osd.5 v2:172.21.15.172:6808/120244 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f287806f860 con 0x7f287806cd80 2023-10-07T08:01:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.830+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 --> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f28940b3190 con 0x7f287806cd80 2023-10-07T08:01:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f288a7fc700 1 -- 172.21.15.7:0/2447742302 <== osd.5 v2:172.21.15.172:6808/120244 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f28940b3190 con 0x7f287806cd80 2023-10-07T08:01:06.832 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T08:01:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T08:01:06.832 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T08:01:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f287806cd80 msgr2=0x7f287806f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6808/120244,v1:172.21.15.172:6809/120244] conn(0x7f287806cd80 0x7f287806f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2878066f30 msgr2=0x7f28780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f2878066f30 0x7f28780693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2890007fd0 tx=0x7f2890007e10 comp rx=0 tx=0).stop 2023-10-07T08:01:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28941585f0 msgr2=0x7f28940b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 --2- 172.21.15.7:0/2447742302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28941585f0 0x7f28940b3780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f288c015000 tx=0x7f288c018000 comp rx=0 tx=0).stop 2023-10-07T08:01:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289a825700 1 -- 172.21.15.7:0/2447742302 reap_dead start 2023-10-07T08:01:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 shutdown_connections 2023-10-07T08:01:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 >> 172.21.15.7:0/2447742302 conn(0x7f2894067a70 msgr2=0x7f289415a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 shutdown_connections 2023-10-07T08:01:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.831+0000 7f289c288700 1 -- 172.21.15.7:0/2447742302 wait complete. 2023-10-07T08:01:06.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-07T08:01:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.977+0000 7fdfaed18700 1 Processor -- start 2023-10-07T08:01:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.977+0000 7fdfaed18700 1 -- start start 2023-10-07T08:01:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.978+0000 7fdfaed18700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdfa81570a0 0x7fdfa80c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfaed18700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa80c3b80 con 0x7fdfa8157470 2023-10-07T08:01:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfaed18700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa80c2f70 con 0x7fdfa806bd30 2023-10-07T08:01:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfaed18700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa80c3250 con 0x7fdfa81570a0 2023-10-07T08:01:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfacab4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdfa81570a0 0x7fdfa80c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfa7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdfa8157470 0x7fdfa806b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47074/0 (socket says 172.21.15.7:47074) 2023-10-07T08:01:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfa7fff700 1 -- 172.21.15.7:0/2441861325 learned_addr learned my addr 172.21.15.7:0/2441861325 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfa77fe700 1 -- 172.21.15.7:0/2441861325 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671559412 0 0) 0x7fdfa80c3b80 con 0x7fdfa8157470 2023-10-07T08:01:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfa77fe700 1 -- 172.21.15.7:0/2441861325 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf94003570 con 0x7fdfa8157470 2023-10-07T08:01:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfacab4700 1 -- 172.21.15.7:0/2441861325 >> v1:172.21.15.172:6789/0 conn(0x7fdfa806bd30 legacy=0x7fdfa80c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfacab4700 1 -- 172.21.15.7:0/2441861325 >> v1:172.21.15.7:6789/0 conn(0x7fdfa8157470 legacy=0x7fdfa806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfacab4700 1 -- 172.21.15.7:0/2441861325 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa80c3530 con 0x7fdfa81570a0 2023-10-07T08:01:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.979+0000 7fdfacab4700 1 --2- 172.21.15.7:0/2441861325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81570a0 0x7fdfa80c8800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdf980098e0 tx=0x7fdf980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd88b0fdbc2c76f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfa77fe700 1 -- 172.21.15.7:0/2441861325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800b640 con 0x7fdfa81570a0 2023-10-07T08:01:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfa77fe700 1 -- 172.21.15.7:0/2441861325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9800b7a0 con 0x7fdfa81570a0 2023-10-07T08:01:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfa77fe700 1 -- 172.21.15.7:0/2441861325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800bb20 con 0x7fdfa81570a0 2023-10-07T08:01:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfaed18700 1 -- 172.21.15.7:0/2441861325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81570a0 msgr2=0x7fdfa80c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfaed18700 1 --2- 172.21.15.7:0/2441861325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81570a0 0x7fdfa80c8800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdf980098e0 tx=0x7fdf980091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfaed18700 1 -- 172.21.15.7:0/2441861325 shutdown_connections 2023-10-07T08:01:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfaed18700 1 --2- 172.21.15.7:0/2441861325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81570a0 0x7fdfa80c8800 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfaed18700 1 -- 172.21.15.7:0/2441861325 >> 172.21.15.7:0/2441861325 conn(0x7fdfa8067ac0 msgr2=0x7fdfa80ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfaed18700 1 -- 172.21.15.7:0/2441861325 shutdown_connections 2023-10-07T08:01:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.980+0000 7fdfaed18700 1 -- 172.21.15.7:0/2441861325 wait complete. 2023-10-07T08:01:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.982+0000 7fdfaed18700 1 Processor -- start 2023-10-07T08:01:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.982+0000 7fdfaed18700 1 -- start start 2023-10-07T08:01:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfaed18700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa806bd30 0x7fdfa813f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfaed18700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa81570a0 0x7fdfa815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfaed18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa8157470 0x7fdfa8139700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfaed18700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80b0e40 con 0x7fdfa8157470 2023-10-07T08:01:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfaed18700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80b1950 con 0x7fdfa81570a0 2023-10-07T08:01:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfaed18700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80b1020 con 0x7fdfa806bd30 2023-10-07T08:01:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfad2b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa8157470 0x7fdfa8139700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfad2b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa8157470 0x7fdfa8139700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45238/0 (socket says 172.21.15.7:45238) 2023-10-07T08:01:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfad2b5700 1 -- 172.21.15.7:0/4220663886 learned_addr learned my addr 172.21.15.7:0/4220663886 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfacab4700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa806bd30 0x7fdfa813f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.983+0000 7fdfa7fff700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa81570a0 0x7fdfa815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfad2b5700 1 -- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa806bd30 msgr2=0x7fdfa813f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfad2b5700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa806bd30 0x7fdfa813f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfad2b5700 1 -- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa81570a0 msgr2=0x7fdfa815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:06.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfad2b5700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa81570a0 0x7fdfa815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:06.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfad2b5700 1 -- 172.21.15.7:0/4220663886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98009420 con 0x7fdfa8157470 2023-10-07T08:01:06.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfad2b5700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa8157470 0x7fdfa8139700 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c840 tx=0x7fdfa0001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfa4ff9700 1 -- 172.21.15.7:0/4220663886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0012630 con 0x7fdfa8157470 2023-10-07T08:01:06.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfa4ff9700 1 -- 172.21.15.7:0/4220663886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa00127b0 con 0x7fdfa8157470 2023-10-07T08:01:06.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa80b1300 con 0x7fdfa8157470 2023-10-07T08:01:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfa4ff9700 1 -- 172.21.15.7:0/4220663886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0012b70 con 0x7fdfa8157470 2023-10-07T08:01:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.984+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa806d1e0 con 0x7fdfa8157470 2023-10-07T08:01:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.985+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa80b1b00 con 0x7fdfa8157470 2023-10-07T08:01:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.986+0000 7fdfa4ff9700 1 -- 172.21.15.7:0/4220663886 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfa001a3a0 con 0x7fdfa8157470 2023-10-07T08:01:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.986+0000 7fdfa4ff9700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdf88066f30 0x7fdf880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.986+0000 7fdfa4ff9700 1 -- 172.21.15.7:0/4220663886 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdfa00842f0 con 0x7fdfa8157470 2023-10-07T08:01:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.986+0000 7fdfacab4700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdf88066f30 0x7fdf880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.987+0000 7fdfacab4700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdf88066f30 0x7fdf880693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf980093f0 tx=0x7fdf98017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:06.990+0000 7fdfa4ff9700 1 -- 172.21.15.7:0/4220663886 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfa80b1b00 con 0x7fdfa8157470 2023-10-07T08:01:07.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.193+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfa80c8f50 con 0x7fdf88066f30 2023-10-07T08:01:07.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.193+0000 7fdfa4ff9700 1 -- 172.21.15.7:0/4220663886 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdfa80c8f50 con 0x7fdf88066f30 2023-10-07T08:01:07.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdf88066f30 msgr2=0x7fdf880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdf88066f30 0x7fdf880693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf980093f0 tx=0x7fdf98017000 comp rx=0 tx=0).stop 2023-10-07T08:01:07.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa8157470 msgr2=0x7fdfa8139700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa8157470 0x7fdfa8139700 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c840 tx=0x7fdfa0001610 comp rx=0 tx=0).stop 2023-10-07T08:01:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 shutdown_connections 2023-10-07T08:01:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fdf88066f30 0x7fdf880693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa8157470 0x7fdfa8139700 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa806bd30 0x7fdfa813f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 --2- 172.21.15.7:0/4220663886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa81570a0 0x7fdfa815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 >> 172.21.15.7:0/4220663886 conn(0x7fdfa8067ac0 msgr2=0x7fdfa813b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 shutdown_connections 2023-10-07T08:01:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.196+0000 7fdfaed18700 1 -- 172.21.15.7:0/4220663886 wait complete. 2023-10-07T08:01:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2023-10-07T08:01:07.211 INFO:tasks.ceph:Scrubbing osd.6 2023-10-07T08:01:07.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-07T08:01:07.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.347+0000 7fec5ef4f700 1 Processor -- start 2023-10-07T08:01:07.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.347+0000 7fec5ef4f700 1 -- start start 2023-10-07T08:01:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.348+0000 7fec5ef4f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fec5813c580 0x7fec5815a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.348+0000 7fec5ef4f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec5806b4f0 0x7fec5806b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.348+0000 7fec5ef4f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec5806be90 con 0x7fec5813c1b0 2023-10-07T08:01:07.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.348+0000 7fec5ef4f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec5815dac0 con 0x7fec5813c580 2023-10-07T08:01:07.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.348+0000 7fec5ef4f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5815df50 con 0x7fec5806b4f0 2023-10-07T08:01:07.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.348+0000 7fec5cceb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fec5813c1b0 0x7fec580c2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47086/0 (socket says 172.21.15.7:47086) 2023-10-07T08:01:07.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.348+0000 7fec5cceb700 1 -- 172.21.15.7:0/2550210903 learned_addr learned my addr 172.21.15.7:0/2550210903 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:07.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.348+0000 7fec57fff700 1 --2- 172.21.15.7:0/2550210903 >> v2:172.21.15.172:3300/0 conn(0x7fec5813c580 0x7fec5815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.348+0000 7fec5d4ec700 1 --2- 172.21.15.7:0/2550210903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec5806b4f0 0x7fec5806b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec577fe700 1 -- 172.21.15.7:0/2550210903 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877585648 0 0) 0x7fec48001110 con 0x7fec5813c1b0 2023-10-07T08:01:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec577fe700 1 -- 172.21.15.7:0/2550210903 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec40003570 con 0x7fec5813c1b0 2023-10-07T08:01:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec57fff700 1 -- 172.21.15.7:0/2550210903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec5806b4f0 msgr2=0x7fec5806b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec57fff700 1 --2- 172.21.15.7:0/2550210903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec5806b4f0 0x7fec5806b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec57fff700 1 -- 172.21.15.7:0/2550210903 >> v1:172.21.15.7:6789/0 conn(0x7fec5813c1b0 legacy=0x7fec580c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec57fff700 1 -- 172.21.15.7:0/2550210903 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5815d190 con 0x7fec5813c580 2023-10-07T08:01:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec57fff700 1 --2- 172.21.15.7:0/2550210903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec5813c580 0x7fec5815a650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0098e0 tx=0x7fec4c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4935d95bb7cd40f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec577fe700 1 -- 172.21.15.7:0/2550210903 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c011630 con 0x7fec5813c580 2023-10-07T08:01:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec577fe700 1 -- 172.21.15.7:0/2550210903 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c011790 con 0x7fec5813c580 2023-10-07T08:01:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.349+0000 7fec577fe700 1 -- 172.21.15.7:0/2550210903 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c011b30 con 0x7fec5813c580 2023-10-07T08:01:07.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.350+0000 7fec5ef4f700 1 -- 172.21.15.7:0/2550210903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec5813c580 msgr2=0x7fec5815a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.350+0000 7fec5ef4f700 1 --2- 172.21.15.7:0/2550210903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec5813c580 0x7fec5815a650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0098e0 tx=0x7fec4c0091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.350+0000 7fec5ef4f700 1 -- 172.21.15.7:0/2550210903 shutdown_connections 2023-10-07T08:01:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.350+0000 7fec5ef4f700 1 --2- 172.21.15.7:0/2550210903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec5806b4f0 0x7fec5806b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.350+0000 7fec5ef4f700 1 --2- 172.21.15.7:0/2550210903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec5813c580 0x7fec5815a650 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.350+0000 7fec5ef4f700 1 -- 172.21.15.7:0/2550210903 >> 172.21.15.7:0/2550210903 conn(0x7fec58067a70 msgr2=0x7fec580ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.350+0000 7fec5ef4f700 1 -- 172.21.15.7:0/2550210903 shutdown_connections 2023-10-07T08:01:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.350+0000 7fec5ef4f700 1 -- 172.21.15.7:0/2550210903 wait complete. 2023-10-07T08:01:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.351+0000 7fec5ef4f700 1 Processor -- start 2023-10-07T08:01:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.351+0000 7fec5ef4f700 1 -- start start 2023-10-07T08:01:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5ef4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5806b4f0 0x7fec580b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5ef4f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec5813c1b0 0x7fec581599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5ef4f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec5813c580 0x7fec58156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5ef4f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec580b1ce0 con 0x7fec5806b4f0 2023-10-07T08:01:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5ef4f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec58159f30 con 0x7fec5813c1b0 2023-10-07T08:01:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5ef4f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec580b10d0 con 0x7fec5813c580 2023-10-07T08:01:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5cceb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5806b4f0 0x7fec580b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5cceb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5806b4f0 0x7fec580b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45244/0 (socket says 172.21.15.7:45244) 2023-10-07T08:01:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5cceb700 1 -- 172.21.15.7:0/1010495022 learned_addr learned my addr 172.21.15.7:0/1010495022 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec57fff700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec5813c1b0 0x7fec581599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.352+0000 7fec5d4ec700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec5813c580 0x7fec58156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec5cceb700 1 -- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec5813c580 msgr2=0x7fec58156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec5cceb700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec5813c580 0x7fec58156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec5cceb700 1 -- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec5813c1b0 msgr2=0x7fec581599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec5cceb700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec5813c1b0 0x7fec581599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec5cceb700 1 -- 172.21.15.7:0/1010495022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c009420 con 0x7fec5806b4f0 2023-10-07T08:01:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec5cceb700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5806b4f0 0x7fec580b37b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fec4800c7c0 tx=0x7fec48007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec54ff9700 1 -- 172.21.15.7:0/1010495022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48005c00 con 0x7fec5806b4f0 2023-10-07T08:01:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec54ff9700 1 -- 172.21.15.7:0/1010495022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec48005d60 con 0x7fec5806b4f0 2023-10-07T08:01:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec580b13b0 con 0x7fec5806b4f0 2023-10-07T08:01:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec54ff9700 1 -- 172.21.15.7:0/1010495022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48007320 con 0x7fec5806b4f0 2023-10-07T08:01:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.353+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec580b1970 con 0x7fec5806b4f0 2023-10-07T08:01:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.354+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec580716c0 con 0x7fec5806b4f0 2023-10-07T08:01:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.355+0000 7fec54ff9700 1 -- 172.21.15.7:0/1010495022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec4801b3b0 con 0x7fec5806b4f0 2023-10-07T08:01:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.355+0000 7fec54ff9700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec44066f30 0x7fec440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.355+0000 7fec54ff9700 1 -- 172.21.15.7:0/1010495022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fec48083630 con 0x7fec5806b4f0 2023-10-07T08:01:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.355+0000 7fec54ff9700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7fec4406cd80 0x7fec4406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.355+0000 7fec54ff9700 1 -- 172.21.15.7:0/1010495022 --> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec4406f860 con 0x7fec4406cd80 2023-10-07T08:01:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.355+0000 7fec54ff9700 1 -- 172.21.15.7:0/1010495022 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec48085c70 con 0x7fec5806b4f0 2023-10-07T08:01:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.355+0000 7fec57fff700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec44066f30 0x7fec440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.355+0000 7fec5d4ec700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7fec4406cd80 0x7fec4406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.356+0000 7fec57fff700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec44066f30 0x7fec440693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec5806f500 tx=0x7fec4c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.356+0000 7fec5d4ec700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7fec4406cd80 0x7fec4406f190 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.357+0000 7fec54ff9700 1 -- 172.21.15.7:0/1010495022 <== osd.6 v2:172.21.15.172:6816/120273 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fec4406f860 con 0x7fec4406cd80 2023-10-07T08:01:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.371+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 --> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fec5806be90 con 0x7fec4406cd80 2023-10-07T08:01:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec54ff9700 1 -- 172.21.15.7:0/1010495022 <== osd.6 v2:172.21.15.172:6816/120273 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fec5806be90 con 0x7fec4406cd80 2023-10-07T08:01:07.373 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T08:01:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T08:01:07.373 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T08:01:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7fec4406cd80 msgr2=0x7fec4406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6816/120273,v1:172.21.15.172:6817/120273] conn(0x7fec4406cd80 0x7fec4406f190 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec44066f30 msgr2=0x7fec440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fec44066f30 0x7fec440693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec5806f500 tx=0x7fec4c017000 comp rx=0 tx=0).stop 2023-10-07T08:01:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5806b4f0 msgr2=0x7fec580b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 --2- 172.21.15.7:0/1010495022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5806b4f0 0x7fec580b37b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fec4800c7c0 tx=0x7fec48007c70 comp rx=0 tx=0).stop 2023-10-07T08:01:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5d4ec700 1 -- 172.21.15.7:0/1010495022 reap_dead start 2023-10-07T08:01:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 shutdown_connections 2023-10-07T08:01:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 >> 172.21.15.7:0/1010495022 conn(0x7fec58067a70 msgr2=0x7fec5815c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 shutdown_connections 2023-10-07T08:01:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.372+0000 7fec5ef4f700 1 -- 172.21.15.7:0/1010495022 wait complete. 2023-10-07T08:01:07.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-07T08:01:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.520+0000 7f97d222c700 1 Processor -- start 2023-10-07T08:01:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.520+0000 7f97d222c700 1 -- start start 2023-10-07T08:01:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97d222c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f97cc155f20 0x7f97cc0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97d222c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc06e9d0 0x7f97cc06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97d222c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97cc06b700 con 0x7f97cc1562f0 2023-10-07T08:01:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97d222c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97cc06aaf0 con 0x7f97cc155f20 2023-10-07T08:01:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97d222c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc06add0 con 0x7f97cc06e9d0 2023-10-07T08:01:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97caffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f97cc1562f0 0x7f97cc06e2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47090/0 (socket says 172.21.15.7:47090) 2023-10-07T08:01:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97caffd700 1 -- 172.21.15.7:0/2048612339 learned_addr learned my addr 172.21.15.7:0/2048612339 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97cb7fe700 1 --2- 172.21.15.7:0/2048612339 >> v2:172.21.15.172:3300/0 conn(0x7f97cc155f20 0x7f97cc0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97cbfff700 1 --2- 172.21.15.7:0/2048612339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc06e9d0 0x7f97cc06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97ca7fc700 1 -- 172.21.15.7:0/2048612339 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698128176 0 0) 0x7f97cc06b700 con 0x7f97cc1562f0 2023-10-07T08:01:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97ca7fc700 1 -- 172.21.15.7:0/2048612339 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b4003570 con 0x7f97cc1562f0 2023-10-07T08:01:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97cb7fe700 1 -- 172.21.15.7:0/2048612339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc06e9d0 msgr2=0x7f97cc06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97cb7fe700 1 --2- 172.21.15.7:0/2048612339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc06e9d0 0x7f97cc06a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97cb7fe700 1 -- 172.21.15.7:0/2048612339 >> v1:172.21.15.7:6789/0 conn(0x7f97cc1562f0 legacy=0x7f97cc06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.521+0000 7f97cb7fe700 1 -- 172.21.15.7:0/2048612339 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc06b0b0 con 0x7f97cc155f20 2023-10-07T08:01:07.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97cb7fe700 1 --2- 172.21.15.7:0/2048612339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97cc155f20 0x7f97cc0c92b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0098e0 tx=0x7f97bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=335b750425519722 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97ca7fc700 1 -- 172.21.15.7:0/2048612339 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc011630 con 0x7f97cc155f20 2023-10-07T08:01:07.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97ca7fc700 1 -- 172.21.15.7:0/2048612339 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc011790 con 0x7f97cc155f20 2023-10-07T08:01:07.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97ca7fc700 1 -- 172.21.15.7:0/2048612339 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc011b30 con 0x7f97cc155f20 2023-10-07T08:01:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97d222c700 1 -- 172.21.15.7:0/2048612339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97cc155f20 msgr2=0x7f97cc0c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97d222c700 1 --2- 172.21.15.7:0/2048612339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97cc155f20 0x7f97cc0c92b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0098e0 tx=0x7f97bc0091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97d222c700 1 -- 172.21.15.7:0/2048612339 shutdown_connections 2023-10-07T08:01:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97d222c700 1 --2- 172.21.15.7:0/2048612339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc06e9d0 0x7f97cc06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97d222c700 1 --2- 172.21.15.7:0/2048612339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97cc155f20 0x7f97cc0c92b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97d222c700 1 -- 172.21.15.7:0/2048612339 >> 172.21.15.7:0/2048612339 conn(0x7f97cc068bd0 msgr2=0x7f97cc0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97d222c700 1 -- 172.21.15.7:0/2048612339 shutdown_connections 2023-10-07T08:01:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.522+0000 7f97d222c700 1 -- 172.21.15.7:0/2048612339 wait complete. 2023-10-07T08:01:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.524+0000 7f97d222c700 1 Processor -- start 2023-10-07T08:01:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.524+0000 7f97d222c700 1 -- start start 2023-10-07T08:01:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.525+0000 7f97d222c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97cc06e9d0 0x7f97cc16a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.525+0000 7f97d222c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc155f20 0x7f97cc0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.525+0000 7f97d222c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc1562f0 0x7f97cc147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.525+0000 7f97d222c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc16b730 con 0x7f97cc1562f0 2023-10-07T08:01:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.525+0000 7f97d222c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0b2300 con 0x7f97cc06e9d0 2023-10-07T08:01:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.525+0000 7f97d222c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc158c50 con 0x7f97cc155f20 2023-10-07T08:01:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.525+0000 7f97cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc1562f0 0x7f97cc147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc1562f0 0x7f97cc147950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45256/0 (socket says 172.21.15.7:45256) 2023-10-07T08:01:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97cbfff700 1 -- 172.21.15.7:0/372756730 learned_addr learned my addr 172.21.15.7:0/372756730 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97caffd700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc155f20 0x7f97cc0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97cb7fe700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97cc06e9d0 0x7f97cc16a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97cbfff700 1 -- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc155f20 msgr2=0x7f97cc0b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97cbfff700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc155f20 0x7f97cc0b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97cbfff700 1 -- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97cc06e9d0 msgr2=0x7f97cc16a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97cbfff700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97cc06e9d0 0x7f97cc16a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97cbfff700 1 -- 172.21.15.7:0/372756730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc009420 con 0x7f97cc1562f0 2023-10-07T08:01:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97cbfff700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc1562f0 0x7f97cc147950 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009e90 tx=0x7f97c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97b37fe700 1 -- 172.21.15.7:0/372756730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400cbc0 con 0x7f97cc1562f0 2023-10-07T08:01:07.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97b37fe700 1 -- 172.21.15.7:0/372756730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c400f040 con 0x7f97cc1562f0 2023-10-07T08:01:07.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc16ae90 con 0x7f97cc1562f0 2023-10-07T08:01:07.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97b37fe700 1 -- 172.21.15.7:0/372756730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4014480 con 0x7f97cc1562f0 2023-10-07T08:01:07.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.526+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc16b450 con 0x7f97cc1562f0 2023-10-07T08:01:07.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.527+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc158a20 con 0x7f97cc1562f0 2023-10-07T08:01:07.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.528+0000 7f97b37fe700 1 -- 172.21.15.7:0/372756730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97c40147f0 con 0x7f97cc1562f0 2023-10-07T08:01:07.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.528+0000 7f97b37fe700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f97b8066f30 0x7f97b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.528+0000 7f97b37fe700 1 -- 172.21.15.7:0/372756730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f97c4084f10 con 0x7f97cc1562f0 2023-10-07T08:01:07.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.528+0000 7f97cb7fe700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f97b8066f30 0x7f97b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.529+0000 7f97cb7fe700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f97b8066f30 0x7f97b80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0091e0 tx=0x7f97bc017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.533+0000 7f97b37fe700 1 -- 172.21.15.7:0/372756730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97cc158a20 con 0x7f97cc1562f0 2023-10-07T08:01:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.735+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f97cc158a20 con 0x7f97b8066f30 2023-10-07T08:01:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.736+0000 7f97b37fe700 1 -- 172.21.15.7:0/372756730 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f97cc158a20 con 0x7f97b8066f30 2023-10-07T08:01:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f97b8066f30 msgr2=0x7f97b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f97b8066f30 0x7f97b80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0091e0 tx=0x7f97bc017000 comp rx=0 tx=0).stop 2023-10-07T08:01:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc1562f0 msgr2=0x7f97cc147950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc1562f0 0x7f97cc147950 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009e90 tx=0x7f97c400b040 comp rx=0 tx=0).stop 2023-10-07T08:01:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 shutdown_connections 2023-10-07T08:01:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f97b8066f30 0x7f97b80693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc1562f0 0x7f97cc147950 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97cc06e9d0 0x7f97cc16a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 --2- 172.21.15.7:0/372756730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97cc155f20 0x7f97cc0b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 >> 172.21.15.7:0/372756730 conn(0x7f97cc068bd0 msgr2=0x7f97cc0c6910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 shutdown_connections 2023-10-07T08:01:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.738+0000 7f97d222c700 1 -- 172.21.15.7:0/372756730 wait complete. 2023-10-07T08:01:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2023-10-07T08:01:07.753 INFO:tasks.ceph:Scrubbing osd.7 2023-10-07T08:01:07.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-07T08:01:07.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.889+0000 7f032d388700 1 Processor -- start 2023-10-07T08:01:07.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.889+0000 7f032d388700 1 -- start start 2023-10-07T08:01:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f032d388700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f03280af640 0x7f032815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f032d388700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280ccbf0 0x7f03280cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f032d388700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032815cf90 con 0x7f0328159330 2023-10-07T08:01:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f032d388700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280b2b30 con 0x7f03280af640 2023-10-07T08:01:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f032d388700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03280cd5b0 con 0x7f03280ccbf0 2023-10-07T08:01:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f0326ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0328159330 0x7f03280afe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47098/0 (socket says 172.21.15.7:47098) 2023-10-07T08:01:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f0326ffd700 1 -- 172.21.15.7:0/1330239911 learned_addr learned my addr 172.21.15.7:0/1330239911 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f03277fe700 1 --2- 172.21.15.7:0/1330239911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280ccbf0 0x7f03280cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f03267fc700 1 --2- 172.21.15.7:0/1330239911 >> v2:172.21.15.172:3300/0 conn(0x7f03280af640 0x7f032815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f0325ffb700 1 -- 172.21.15.7:0/1330239911 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562045171 0 0) 0x7f032815cf90 con 0x7f0328159330 2023-10-07T08:01:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.890+0000 7f0325ffb700 1 -- 172.21.15.7:0/1330239911 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0310003570 con 0x7f0328159330 2023-10-07T08:01:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.891+0000 7f03267fc700 1 -- 172.21.15.7:0/1330239911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280ccbf0 msgr2=0x7f03280cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.891+0000 7f03267fc700 1 --2- 172.21.15.7:0/1330239911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280ccbf0 0x7f03280cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.891+0000 7f03267fc700 1 -- 172.21.15.7:0/1330239911 >> v1:172.21.15.7:6789/0 conn(0x7f0328159330 legacy=0x7f03280afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.891+0000 7f03267fc700 1 -- 172.21.15.7:0/1330239911 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032815d550 con 0x7f03280af640 2023-10-07T08:01:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.891+0000 7f03277fe700 1 --2- 172.21.15.7:0/1330239911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280ccbf0 0x7f03280cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.891+0000 7f03267fc700 1 --2- 172.21.15.7:0/1330239911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280af640 0x7f032815d9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f031c009910 tx=0x7f031c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b5e271f1779506 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.891+0000 7f0325ffb700 1 -- 172.21.15.7:0/1330239911 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c011630 con 0x7f03280af640 2023-10-07T08:01:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.891+0000 7f0325ffb700 1 -- 172.21.15.7:0/1330239911 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c011790 con 0x7f03280af640 2023-10-07T08:01:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.891+0000 7f0325ffb700 1 -- 172.21.15.7:0/1330239911 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c011b30 con 0x7f03280af640 2023-10-07T08:01:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.892+0000 7f032d388700 1 -- 172.21.15.7:0/1330239911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280af640 msgr2=0x7f032815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.892+0000 7f032d388700 1 --2- 172.21.15.7:0/1330239911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280af640 0x7f032815d9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f031c009910 tx=0x7f031c0091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.892+0000 7f032d388700 1 -- 172.21.15.7:0/1330239911 shutdown_connections 2023-10-07T08:01:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.892+0000 7f032d388700 1 --2- 172.21.15.7:0/1330239911 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280ccbf0 0x7f03280cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.892+0000 7f032d388700 1 --2- 172.21.15.7:0/1330239911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280af640 0x7f032815d9e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.892+0000 7f032d388700 1 -- 172.21.15.7:0/1330239911 >> 172.21.15.7:0/1330239911 conn(0x7f0328069f20 msgr2=0x7f032806e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.892+0000 7f032d388700 1 -- 172.21.15.7:0/1330239911 shutdown_connections 2023-10-07T08:01:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.892+0000 7f032d388700 1 -- 172.21.15.7:0/1330239911 wait complete. 2023-10-07T08:01:07.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.893+0000 7f032d388700 1 Processor -- start 2023-10-07T08:01:07.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.893+0000 7f032d388700 1 -- start start 2023-10-07T08:01:07.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f032d388700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280af640 0x7f03280c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f032d388700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03280ccbf0 0x7f03280c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f032d388700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0328159330 0x7f0328158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f032d388700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280c3d40 con 0x7f03280ccbf0 2023-10-07T08:01:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f032d388700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280c4300 con 0x7f0328159330 2023-10-07T08:01:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f032d388700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03280c84d0 con 0x7f03280af640 2023-10-07T08:01:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03267fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03280ccbf0 0x7f03280c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03267fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03280ccbf0 0x7f03280c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45272/0 (socket says 172.21.15.7:45272) 2023-10-07T08:01:07.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03267fc700 1 -- 172.21.15.7:0/2129861620 learned_addr learned my addr 172.21.15.7:0/2129861620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:07.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f0326ffd700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280af640 0x7f03280c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03277fe700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0328159330 0x7f0328158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03267fc700 1 -- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280af640 msgr2=0x7f03280c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03267fc700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280af640 0x7f03280c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03267fc700 1 -- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0328159330 msgr2=0x7f0328158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03267fc700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0328159330 0x7f0328158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03267fc700 1 -- 172.21.15.7:0/2129861620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0320009a20 con 0x7f03280ccbf0 2023-10-07T08:01:07.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f0326ffd700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03280af640 0x7f03280c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:01:07.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.894+0000 7f03277fe700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0328159330 0x7f0328158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T08:01:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.895+0000 7f03267fc700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03280ccbf0 0x7f03280c9030 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0328068ec0 tx=0x7f031c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.895+0000 7f030effd700 1 -- 172.21.15.7:0/2129861620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c011ea0 con 0x7f03280ccbf0 2023-10-07T08:01:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.895+0000 7f030effd700 1 -- 172.21.15.7:0/2129861620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c011b10 con 0x7f03280ccbf0 2023-10-07T08:01:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.895+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031c009420 con 0x7f03280ccbf0 2023-10-07T08:01:07.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.895+0000 7f030effd700 1 -- 172.21.15.7:0/2129861620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c01a680 con 0x7f03280ccbf0 2023-10-07T08:01:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.895+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03280c3f90 con 0x7f03280ccbf0 2023-10-07T08:01:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.896+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f03280716c0 con 0x7f03280ccbf0 2023-10-07T08:01:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.896+0000 7f030effd700 1 -- 172.21.15.7:0/2129861620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f031c01aa70 con 0x7f03280ccbf0 2023-10-07T08:01:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.897+0000 7f030effd700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0314066f30 0x7f03140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.897+0000 7f030effd700 1 -- 172.21.15.7:0/2129861620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f031c086090 con 0x7f03280ccbf0 2023-10-07T08:01:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.897+0000 7f030effd700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f031406cd80 0x7f031406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.897+0000 7f030effd700 1 -- 172.21.15.7:0/2129861620 --> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f031406f860 con 0x7f031406cd80 2023-10-07T08:01:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.897+0000 7f030effd700 1 -- 172.21.15.7:0/2129861620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f031c0885f0 con 0x7f03280ccbf0 2023-10-07T08:01:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.897+0000 7f0326ffd700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0314066f30 0x7f03140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.897+0000 7f03277fe700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f031406cd80 0x7f031406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.898+0000 7f0326ffd700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0314066f30 0x7f03140693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f03180058d0 tx=0x7f0318009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.898+0000 7f03277fe700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f031406cd80 0x7f031406f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.898+0000 7f030effd700 1 -- 172.21.15.7:0/2129861620 <== osd.7 v2:172.21.15.172:6824/120274 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f031406f860 con 0x7f031406cd80 2023-10-07T08:01:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.913+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 --> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f03280cc4f0 con 0x7f031406cd80 2023-10-07T08:01:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f030effd700 1 -- 172.21.15.7:0/2129861620 <== osd.7 v2:172.21.15.172:6824/120274 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f03280cc4f0 con 0x7f031406cd80 2023-10-07T08:01:07.915 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-10-07T08:01:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T08:01:07.915 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-10-07T08:01:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f031406cd80 msgr2=0x7f031406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6824/120274,v1:172.21.15.172:6825/120274] conn(0x7f031406cd80 0x7f031406f190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0314066f30 msgr2=0x7f03140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f0314066f30 0x7f03140693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f03180058d0 tx=0x7f0318009040 comp rx=0 tx=0).stop 2023-10-07T08:01:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03280ccbf0 msgr2=0x7f03280c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 --2- 172.21.15.7:0/2129861620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03280ccbf0 0x7f03280c9030 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0328068ec0 tx=0x7f031c017000 comp rx=0 tx=0).stop 2023-10-07T08:01:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f03277fe700 1 -- 172.21.15.7:0/2129861620 reap_dead start 2023-10-07T08:01:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 shutdown_connections 2023-10-07T08:01:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 >> 172.21.15.7:0/2129861620 conn(0x7f0328069f20 msgr2=0x7f03280cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 shutdown_connections 2023-10-07T08:01:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:07.914+0000 7f032d388700 1 -- 172.21.15.7:0/2129861620 wait complete. 2023-10-07T08:01:07.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-07T08:01:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.061+0000 7ff4ee212700 1 Processor -- start 2023-10-07T08:01:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.061+0000 7ff4ee212700 1 -- start start 2023-10-07T08:01:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4ee212700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4e81592c0 0x7ff4e806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4ee212700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff4e80cb290 0x7ff4e806a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4ee212700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e806bb70 con 0x7ff4e8159690 2023-10-07T08:01:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4ee212700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e80cda10 con 0x7ff4e81592c0 2023-10-07T08:01:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4ee212700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e806af60 con 0x7ff4e80cb290 2023-10-07T08:01:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4e81592c0 0x7ff4e806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e6ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff4e8159690 0x7ff4e80cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47102/0 (socket says 172.21.15.7:47102) 2023-10-07T08:01:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4e81592c0 0x7ff4e806a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45276/0 (socket says 172.21.15.7:45276) 2023-10-07T08:01:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e6ffd700 1 -- 172.21.15.7:0/803651319 learned_addr learned my addr 172.21.15.7:0/803651319 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e7fff700 1 --2- 172.21.15.7:0/803651319 >> v2:172.21.15.172:3300/0 conn(0x7ff4e80cb290 0x7ff4e806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e77fe700 1 -- 172.21.15.7:0/803651319 >> v2:172.21.15.172:3300/0 conn(0x7ff4e80cb290 msgr2=0x7ff4e806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e77fe700 1 --2- 172.21.15.7:0/803651319 >> v2:172.21.15.172:3300/0 conn(0x7ff4e80cb290 0x7ff4e806a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e77fe700 1 -- 172.21.15.7:0/803651319 >> v1:172.21.15.7:6789/0 conn(0x7ff4e8159690 legacy=0x7ff4e80cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e67fc700 1 -- 172.21.15.7:0/803651319 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058985369 0 0) 0x7ff4e806bb70 con 0x7ff4e8159690 2023-10-07T08:01:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.062+0000 7ff4e77fe700 1 -- 172.21.15.7:0/803651319 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e806b240 con 0x7ff4e81592c0 2023-10-07T08:01:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4e77fe700 1 --2- 172.21.15.7:0/803651319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e81592c0 0x7ff4e806a3f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80afaf0 tx=0x7ff4d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730632df6bd4d156 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4e67fc700 1 -- 172.21.15.7:0/803651319 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000b6b0 con 0x7ff4e81592c0 2023-10-07T08:01:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4e67fc700 1 -- 172.21.15.7:0/803651319 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d000b810 con 0x7ff4e81592c0 2023-10-07T08:01:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4e67fc700 1 -- 172.21.15.7:0/803651319 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000bbb0 con 0x7ff4e81592c0 2023-10-07T08:01:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4ee212700 1 -- 172.21.15.7:0/803651319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e81592c0 msgr2=0x7ff4e806a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4ee212700 1 --2- 172.21.15.7:0/803651319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e81592c0 0x7ff4e806a3f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80afaf0 tx=0x7ff4d00091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4ee212700 1 -- 172.21.15.7:0/803651319 shutdown_connections 2023-10-07T08:01:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4ee212700 1 --2- 172.21.15.7:0/803651319 >> v2:172.21.15.172:3300/0 conn(0x7ff4e80cb290 0x7ff4e806a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4ee212700 1 --2- 172.21.15.7:0/803651319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e81592c0 0x7ff4e806a3f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4ee212700 1 -- 172.21.15.7:0/803651319 >> 172.21.15.7:0/803651319 conn(0x7ff4e8068bd0 msgr2=0x7ff4e80ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4ee212700 1 -- 172.21.15.7:0/803651319 shutdown_connections 2023-10-07T08:01:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.063+0000 7ff4ee212700 1 -- 172.21.15.7:0/803651319 wait complete. 2023-10-07T08:01:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.065+0000 7ff4ee212700 1 Processor -- start 2023-10-07T08:01:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.065+0000 7ff4ee212700 1 -- start start 2023-10-07T08:01:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4ee212700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e80cb290 0x7ff4e8152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4ee212700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e81592c0 0x7ff4e81516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4ee212700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8159690 0x7ff4e8154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4ee212700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e816a9f0 con 0x7ff4e8159690 2023-10-07T08:01:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4ee212700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e8151f40 con 0x7ff4e80cb290 2023-10-07T08:01:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4ee212700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e8152220 con 0x7ff4e81592c0 2023-10-07T08:01:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8159690 0x7ff4e8154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8159690 0x7ff4e8154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45288/0 (socket says 172.21.15.7:45288) 2023-10-07T08:01:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e7fff700 1 -- 172.21.15.7:0/674841980 learned_addr learned my addr 172.21.15.7:0/674841980 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e77fe700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e80cb290 0x7ff4e8152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e6ffd700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e81592c0 0x7ff4e81516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e7fff700 1 -- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e81592c0 msgr2=0x7ff4e81516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e7fff700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e81592c0 0x7ff4e81516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e7fff700 1 -- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e80cb290 msgr2=0x7ff4e8152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e7fff700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e80cb290 0x7ff4e8152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e7fff700 1 -- 172.21.15.7:0/674841980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d0009420 con 0x7ff4e8159690 2023-10-07T08:01:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4e7fff700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8159690 0x7ff4e8154a70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0007f20 tx=0x7ff4e0009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.066+0000 7ff4df7fe700 1 -- 172.21.15.7:0/674841980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e000eb90 con 0x7ff4e8159690 2023-10-07T08:01:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.067+0000 7ff4df7fe700 1 -- 172.21.15.7:0/674841980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e000ed10 con 0x7ff4e8159690 2023-10-07T08:01:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.067+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e8152500 con 0x7ff4e8159690 2023-10-07T08:01:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.067+0000 7ff4df7fe700 1 -- 172.21.15.7:0/674841980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0014670 con 0x7ff4e8159690 2023-10-07T08:01:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.067+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e8151c00 con 0x7ff4e8159690 2023-10-07T08:01:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.068+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e8153240 con 0x7ff4e8159690 2023-10-07T08:01:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.068+0000 7ff4df7fe700 1 -- 172.21.15.7:0/674841980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4e00149e0 con 0x7ff4e8159690 2023-10-07T08:01:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.069+0000 7ff4df7fe700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff4c8066f30 0x7ff4c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.069+0000 7ff4df7fe700 1 -- 172.21.15.7:0/674841980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff4e0084830 con 0x7ff4e8159690 2023-10-07T08:01:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.069+0000 7ff4e77fe700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff4c8066f30 0x7ff4c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.070+0000 7ff4e77fe700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff4c8066f30 0x7ff4c80693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00093f0 tx=0x7ff4d0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.073+0000 7ff4df7fe700 1 -- 172.21.15.7:0/674841980 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4e0051da0 con 0x7ff4e8159690 2023-10-07T08:01:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.274+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4e80716c0 con 0x7ff4c8066f30 2023-10-07T08:01:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.275+0000 7ff4df7fe700 1 -- 172.21.15.7:0/674841980 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff4e80716c0 con 0x7ff4c8066f30 2023-10-07T08:01:08.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff4c8066f30 msgr2=0x7ff4c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff4c8066f30 0x7ff4c80693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00093f0 tx=0x7ff4d0017000 comp rx=0 tx=0).stop 2023-10-07T08:01:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8159690 msgr2=0x7ff4e8154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8159690 0x7ff4e8154a70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0007f20 tx=0x7ff4e0009860 comp rx=0 tx=0).stop 2023-10-07T08:01:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 shutdown_connections 2023-10-07T08:01:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7ff4c8066f30 0x7ff4c80693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8159690 0x7ff4e8154a70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e80cb290 0x7ff4e8152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 --2- 172.21.15.7:0/674841980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e81592c0 0x7ff4e81516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 >> 172.21.15.7:0/674841980 conn(0x7ff4e8068bd0 msgr2=0x7ff4e80cab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 shutdown_connections 2023-10-07T08:01:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.277+0000 7ff4ee212700 1 -- 172.21.15.7:0/674841980 wait complete. 2023-10-07T08:01:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2023-10-07T08:01:08.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T08:01:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.424+0000 7fb8fbf9c700 1 Processor -- start 2023-10-07T08:01:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.425+0000 7fb8fbf9c700 1 -- start start 2023-10-07T08:01:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.425+0000 7fb8fbf9c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f4159690 0x7fb8f40cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.425+0000 7fb8fbf9c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8f40cb0c0 0x7fb8f40a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.425+0000 7fb8fbf9c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f406bb70 con 0x7fb8f41592c0 2023-10-07T08:01:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.425+0000 7fb8fbf9c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f40cda10 con 0x7fb8f40cb0c0 2023-10-07T08:01:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.426+0000 7fb8fbf9c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f406af60 con 0x7fb8f4159690 2023-10-07T08:01:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.426+0000 7fb8fa539700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8f40cb0c0 0x7fb8f40a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.426+0000 7fb8f9537700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f4159690 0x7fb8f40cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.426+0000 7fb8f9d38700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb8f41592c0 0x7fb8f406a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:43508/0 (socket says 172.21.15.7:43508) 2023-10-07T08:01:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.426+0000 7fb8f9d38700 1 -- 172.21.15.7:0/637990126 learned_addr learned my addr 172.21.15.7:0/637990126 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.426+0000 7fb8fa539700 1 -- 172.21.15.7:0/637990126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f4159690 msgr2=0x7fb8f40cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.426+0000 7fb8fa539700 1 --2- 172.21.15.7:0/637990126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f4159690 0x7fb8f40cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.426+0000 7fb8fa539700 1 -- 172.21.15.7:0/637990126 >> v1:172.21.15.172:6789/0 conn(0x7fb8f41592c0 legacy=0x7fb8f406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.426+0000 7fb8fa539700 1 -- 172.21.15.7:0/637990126 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f406b240 con 0x7fb8f40cb0c0 2023-10-07T08:01:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8fa539700 1 --2- 172.21.15.7:0/637990126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f40cb0c0 0x7fb8f40a9a40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009e60 tx=0x7fb8ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b0dce7b22cf61a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8f8d36700 1 -- 172.21.15.7:0/637990126 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00e070 con 0x7fb8f40cb0c0 2023-10-07T08:01:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8f8d36700 1 -- 172.21.15.7:0/637990126 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec00ba60 con 0x7fb8f40cb0c0 2023-10-07T08:01:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8f8d36700 1 -- 172.21.15.7:0/637990126 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00be20 con 0x7fb8f40cb0c0 2023-10-07T08:01:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/637990126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f40cb0c0 msgr2=0x7fb8f40a9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8fbf9c700 1 --2- 172.21.15.7:0/637990126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f40cb0c0 0x7fb8f40a9a40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009e60 tx=0x7fb8ec009780 comp rx=0 tx=0).stop 2023-10-07T08:01:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/637990126 shutdown_connections 2023-10-07T08:01:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8fbf9c700 1 --2- 172.21.15.7:0/637990126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f40cb0c0 0x7fb8f40a9a40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8fbf9c700 1 --2- 172.21.15.7:0/637990126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f4159690 0x7fb8f40cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/637990126 >> 172.21.15.7:0/637990126 conn(0x7fb8f4068bd0 msgr2=0x7fb8f40ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/637990126 shutdown_connections 2023-10-07T08:01:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.427+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/637990126 wait complete. 2023-10-07T08:01:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.429+0000 7fb8fbf9c700 1 Processor -- start 2023-10-07T08:01:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.429+0000 7fb8fbf9c700 1 -- start start 2023-10-07T08:01:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.429+0000 7fb8fbf9c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f40cb0c0 0x7fb8f4159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.429+0000 7fb8fbf9c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f41592c0 0x7fb8f41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.429+0000 7fb8fbf9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f4159690 0x7fb8f4158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.429+0000 7fb8fbf9c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40ae0e0 con 0x7fb8f4159690 2023-10-07T08:01:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.429+0000 7fb8fbf9c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f415e190 con 0x7fb8f41592c0 2023-10-07T08:01:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.429+0000 7fb8fbf9c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40ade00 con 0x7fb8f40cb0c0 2023-10-07T08:01:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fa539700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f4159690 0x7fb8f4158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fa539700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f4159690 0x7fb8f4158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45290/0 (socket says 172.21.15.7:45290) 2023-10-07T08:01:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fa539700 1 -- 172.21.15.7:0/4108204834 learned_addr learned my addr 172.21.15.7:0/4108204834 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8f9d38700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f40cb0c0 0x7fb8f4159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8f9537700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f41592c0 0x7fb8f41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fa539700 1 -- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f40cb0c0 msgr2=0x7fb8f4159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fa539700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f40cb0c0 0x7fb8f4159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fa539700 1 -- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f41592c0 msgr2=0x7fb8f41584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fa539700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f41592c0 0x7fb8f41584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fa539700 1 -- 172.21.15.7:0/4108204834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec009a20 con 0x7fb8f4159690 2023-10-07T08:01:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fa539700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f4159690 0x7fb8f4158a30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4069020 tx=0x7fb8ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8e9ffb700 1 -- 172.21.15.7:0/4108204834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec012d60 con 0x7fb8f4159690 2023-10-07T08:01:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8e9ffb700 1 -- 172.21.15.7:0/4108204834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec01e430 con 0x7fb8f4159690 2023-10-07T08:01:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f415db40 con 0x7fb8f4159690 2023-10-07T08:01:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.430+0000 7fb8e9ffb700 1 -- 172.21.15.7:0/4108204834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec01e7f0 con 0x7fb8f4159690 2023-10-07T08:01:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.431+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f415a2f0 con 0x7fb8f4159690 2023-10-07T08:01:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.432+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f415d9a0 con 0x7fb8f4159690 2023-10-07T08:01:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.432+0000 7fb8e9ffb700 1 -- 172.21.15.7:0/4108204834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8ec01ebe0 con 0x7fb8f4159690 2023-10-07T08:01:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.432+0000 7fb8e9ffb700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb8d8066f30 0x7fb8d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.432+0000 7fb8e9ffb700 1 -- 172.21.15.7:0/4108204834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb8ec0864e0 con 0x7fb8f4159690 2023-10-07T08:01:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.433+0000 7fb8f9d38700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb8d8066f30 0x7fb8d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.433+0000 7fb8f9d38700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb8d8066f30 0x7fb8d80693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00058d0 tx=0x7fb8f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.437+0000 7fb8e9ffb700 1 -- 172.21.15.7:0/4108204834 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8f415d9a0 con 0x7fb8f4159690 2023-10-07T08:01:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.637+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8f415d9a0 con 0x7fb8d8066f30 2023-10-07T08:01:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.641+0000 7fb8e9ffb700 1 -- 172.21.15.7:0/4108204834 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50570 (secure 0 0 0) 0x7fb8f415d9a0 con 0x7fb8d8066f30 2023-10-07T08:01:08.642 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:01:08.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb8d8066f30 msgr2=0x7fb8d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb8d8066f30 0x7fb8d80693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00058d0 tx=0x7fb8f0009040 comp rx=0 tx=0).stop 2023-10-07T08:01:08.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f4159690 msgr2=0x7fb8f4158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:08.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f4159690 0x7fb8f4158a30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4069020 tx=0x7fb8ec018000 comp rx=0 tx=0).stop 2023-10-07T08:01:08.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 shutdown_connections 2023-10-07T08:01:08.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb8d8066f30 0x7fb8d80693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8f4159690 0x7fb8f4158a30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8f40cb0c0 0x7fb8f4159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 --2- 172.21.15.7:0/4108204834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f41592c0 0x7fb8f41584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:08.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 >> 172.21.15.7:0/4108204834 conn(0x7fb8f4068bd0 msgr2=0x7fb8f406a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:08.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 shutdown_connections 2023-10-07T08:01:08.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:08.643+0000 7fb8fbf9c700 1 -- 172.21.15.7:0/4108204834 wait complete. 2023-10-07T08:01:08.646 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T08:01:08.657 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2023-10-07T08:01:08.071506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2466344,"kb_used_data":6120,"kb_used_omap":0,"kb_used_meta":2460160,"kb_avail":752508376,"statfs":{"total":773094113280,"available":770568577024,"internally_reserved":0,"allocated":6266880,"data_stored":3278608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2519203840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.635758"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-07T08:00:56.432782+0000","last_change":"2023-10-07T07:57:45.954034+0000","last_active":"2023-10-07T08:00:56.432782+0000","last_peered":"2023-10-07T08:00:56.432782+0000","last_clean":"2023-10-07T08:00:56.432782+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T08:00:56.432782+0000","last_undegraded":"2023-10-07T08:00:56.432782+0000","last_fullsized":"2023-10-07T08:00:56.432782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T07:57:44.662724+0000","last_clean_scrub_stamp":"2023-10-07T07:57:44.662724+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:40.725105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306100,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065740,"statfs":{"total":96636764160,"available":96323317760,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63500000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.439,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54300000000000004}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.54700000000000004,"15min":0.55500000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63500000000000001,"5min":1.127,"15min":1.127},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58299999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.57699999999999996}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76900000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58999999999999997}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.626,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70599999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68799999999999994}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.437,"15min":0.45400000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60299999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48799999999999999,"15min":0.51400000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.621}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.50900000000000001,"15min":0.52500000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.53500000000000003,"15min":0.55600000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":1.048,"15min":1.048},"last":0.66000000000000003}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.53400000000000003,"15min":0.55500000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67700000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.57999999999999996,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.70399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73799999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.60099999999999998,"15min":0.61599999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.60799999999999998}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.75700000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.63600000000000001,"15min":0.68100000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66300000000000003,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.58999999999999997}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.69799999999999995,"15min":0.77200000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69099999999999995,"5min":2.653,"15min":2.653},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70999999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66500000000000004,"15min":0.72699999999999998},"min":{"1min":0.314,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67600000000000005,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48799999999999999,"15min":0.504},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.625,"5min":0.878,"15min":0.878},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.44,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64400000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.624},{"interface":"front","average":{"1min":0.498,"5min":0.58499999999999996,"15min":0.60199999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66500000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310776,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061064,"statfs":{"total":96636764160,"available":96318529536,"internally_reserved":0,"allocated":1228800,"data_stored":852602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.443,"15min":0.46899999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53400000000000003,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.56699999999999995,"15min":0.63800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52600000000000002,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.42199999999999999}]},{"osd":2,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.66400000000000003,"15min":0.77600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78100000000000003,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.54400000000000004,"15min":0.57899999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.376}]},{"osd":3,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71499999999999997,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.59099999999999997,"15min":0.64700000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70799999999999996,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.47099999999999997}]},{"osd":4,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65400000000000003,"15min":0.66700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":1.452,"15min":1.452},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.996,"5min":3.3929999999999998,"15min":3.3929999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":1.546,"15min":1.546},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.016,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.81299999999999994,"15min":0.87},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96799999999999997,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.76100000000000001}]},{"osd":7,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.82199999999999995,"15min":0.879},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0309999999999999,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.78500000000000003,"5min":1.585,"15min":1.585},"last":0.69799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.41199999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.70999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.378}]},{"osd":2,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42999999999999999,"15min":0.44},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70099999999999996,"5min":1.298,"15min":1.298},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.33400000000000002}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.71099999999999997,"5min":1.032,"15min":1.032},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.141,"15min":1.141},"last":0.35499999999999998}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88800000000000001,"5min":1.218,"15min":1.218},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":0.69099999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82799999999999996,"5min":1.375,"15min":1.375},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86099999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.401,"15min":1.401},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.69499999999999995,"15min":0.72299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90200000000000002,"5min":1.462,"15min":1.462},"last":0.628}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83699999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.69499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.438},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.39400000000000002,"15min":0.39600000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.45700000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.374}]},{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.51100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.497}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.47399999999999998}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69499999999999995}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.026,"5min":1.51,"15min":1.51},"last":0.61199999999999999}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.88,"15min":0.88},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.625}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.68600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.84499999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.68100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.41699999999999998,"15min":0.437},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.53100000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42599999999999999,"15min":0.435},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64100000000000001,"5min":1.016,"15min":1.016},"last":0.35799999999999998}]},{"osd":1,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.53400000000000003,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.31},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.624,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.49399999999999999,"15min":0.51700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67900000000000005,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.60099999999999998,"5min":1.323,"15min":1.323},"last":0.34200000000000003}]},{"osd":4,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93999999999999995,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96899999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.496}]},{"osd":5,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.327},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.78800000000000003,"15min":0.83799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.751,"15min":0.78700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.78200000000000003,"15min":0.81999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.018,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.72099999999999997,"15min":0.74299999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.89300000000000002,"5min":1.528,"15min":1.528},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0369999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.67700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.65400000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45400000000000001}]},{"osd":2,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.441}]},{"osd":3,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55800000000000005}]},{"osd":5,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77400000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.622}]},{"osd":6,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.503},{"interface":"front","average":{"1min":0.505,"5min":0.56399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":0.878},"last":0.48799999999999999}]},{"osd":7,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80600000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.64600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298488,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073352,"statfs":{"total":96636764160,"available":96331112448,"internally_reserved":0,"allocated":1228800,"data_stored":852602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.60999999999999999,"15min":0.65600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58999999999999997,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.626},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.65400000000000003,"15min":0.70699999999999996},"min":{"1min":0.35599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":3.375,"15min":3.375},"last":0.55100000000000005}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.61699999999999999,"15min":0.66700000000000004},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64300000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.503},{"interface":"front","average":{"1min":0.442,"5min":0.62,"15min":0.66800000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.71499999999999997,"15min":0.79900000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":5.2699999999999996,"15min":5.2699999999999996},"last":0.64400000000000002}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89900000000000002,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.91500000000000004,"15min":1.0860000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":10.644,"15min":10.644},"last":0.61399999999999999}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.54300000000000004,"15min":0.58299999999999996},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.434},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57399999999999995,"5min":1.242,"15min":1.242},"last":0.67900000000000005}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.53600000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.64000000000000001,"15min":0.68899999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63600000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.59199999999999997,"15min":0.61399999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.75600000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65100000000000002,"5min":1.123,"15min":1.123},"last":0.65700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T08:01:08.663 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-07T07:57:44.662724+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=7, tm_hour=7, tm_min=57, tm_sec=44, tm_wday=5, tm_yday=280, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=7, tm_hour=8, tm_min=1, tm_sec=2, tm_wday=5, tm_yday=280, tm_isdst=0) 2023-10-07T08:01:08.664 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-07T08:01:28.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T08:01:28.236 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T08:01:28.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T08:01:28.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.805+0000 7fc629fc7700 1 Processor -- start 2023-10-07T08:01:28.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.805+0000 7fc629fc7700 1 -- start start 2023-10-07T08:01:28.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc629fc7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6241570a0 0x7fc6240c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:28.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc629fc7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc624157470 0x7fc62406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:28.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc629fc7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6240c3b80 con 0x7fc62406bb60 2023-10-07T08:01:28.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc629fc7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62406bfe0 con 0x7fc6241570a0 2023-10-07T08:01:28.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc629fc7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6240c2f70 con 0x7fc624157470 2023-10-07T08:01:28.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc623fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc62406bb60 0x7fc62406cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41612/0 (socket says 172.21.15.7:41612) 2023-10-07T08:01:28.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc623fff700 1 -- 172.21.15.7:0/3658810698 learned_addr learned my addr 172.21.15.7:0/3658810698 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:28.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc6237fe700 1 --2- 172.21.15.7:0/3658810698 >> v2:172.21.15.172:3300/0 conn(0x7fc6241570a0 0x7fc6240c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:28.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc622ffd700 1 --2- 172.21.15.7:0/3658810698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc624157470 0x7fc62406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:28.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc6227fc700 1 -- 172.21.15.7:0/3658810698 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055475380 0 0) 0x7fc6240c3b80 con 0x7fc62406bb60 2023-10-07T08:01:28.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.806+0000 7fc6227fc700 1 -- 172.21.15.7:0/3658810698 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc610003570 con 0x7fc62406bb60 2023-10-07T08:01:28.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.807+0000 7fc6237fe700 1 -- 172.21.15.7:0/3658810698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc624157470 msgr2=0x7fc62406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:28.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.807+0000 7fc6237fe700 1 --2- 172.21.15.7:0/3658810698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc624157470 0x7fc62406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:28.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.807+0000 7fc6237fe700 1 -- 172.21.15.7:0/3658810698 >> v1:172.21.15.7:6789/0 conn(0x7fc62406bb60 legacy=0x7fc62406cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:28.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.807+0000 7fc6237fe700 1 -- 172.21.15.7:0/3658810698 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6240c3250 con 0x7fc6241570a0 2023-10-07T08:01:28.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.807+0000 7fc622ffd700 1 --2- 172.21.15.7:0/3658810698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc624157470 0x7fc62406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:28.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.807+0000 7fc6237fe700 1 --2- 172.21.15.7:0/3658810698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6241570a0 0x7fc6240c8800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6140098e0 tx=0x7fc6140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc193653e64acb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:28.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.807+0000 7fc6227fc700 1 -- 172.21.15.7:0/3658810698 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614011630 con 0x7fc6241570a0 2023-10-07T08:01:28.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.807+0000 7fc6227fc700 1 -- 172.21.15.7:0/3658810698 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc614011790 con 0x7fc6241570a0 2023-10-07T08:01:28.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.807+0000 7fc6227fc700 1 -- 172.21.15.7:0/3658810698 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614011b30 con 0x7fc6241570a0 2023-10-07T08:01:28.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.808+0000 7fc629fc7700 1 -- 172.21.15.7:0/3658810698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6241570a0 msgr2=0x7fc6240c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:28.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.808+0000 7fc629fc7700 1 --2- 172.21.15.7:0/3658810698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6241570a0 0x7fc6240c8800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6140098e0 tx=0x7fc6140091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:28.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.808+0000 7fc629fc7700 1 -- 172.21.15.7:0/3658810698 shutdown_connections 2023-10-07T08:01:28.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.808+0000 7fc629fc7700 1 --2- 172.21.15.7:0/3658810698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6241570a0 0x7fc6240c8800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:28.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.808+0000 7fc629fc7700 1 --2- 172.21.15.7:0/3658810698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc624157470 0x7fc62406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:28.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.808+0000 7fc629fc7700 1 -- 172.21.15.7:0/3658810698 >> 172.21.15.7:0/3658810698 conn(0x7fc624067ac0 msgr2=0x7fc6240ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:28.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.808+0000 7fc629fc7700 1 -- 172.21.15.7:0/3658810698 shutdown_connections 2023-10-07T08:01:28.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.808+0000 7fc629fc7700 1 -- 172.21.15.7:0/3658810698 wait complete. 2023-10-07T08:01:28.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.809+0000 7fc629fc7700 1 Processor -- start 2023-10-07T08:01:28.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.809+0000 7fc629fc7700 1 -- start start 2023-10-07T08:01:28.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc629fc7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62406bb60 0x7fc6240c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc629fc7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6241570a0 0x7fc624152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc629fc7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc624157470 0x7fc62413c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc629fc7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241551d0 con 0x7fc62406bb60 2023-10-07T08:01:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc629fc7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241548e0 con 0x7fc624157470 2023-10-07T08:01:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc629fc7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62413f1c0 con 0x7fc6241570a0 2023-10-07T08:01:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc6237fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62406bb60 0x7fc6240c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc6237fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62406bb60 0x7fc6240c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55966/0 (socket says 172.21.15.7:55966) 2023-10-07T08:01:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc6237fe700 1 -- 172.21.15.7:0/2102052734 learned_addr learned my addr 172.21.15.7:0/2102052734 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc622ffd700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6241570a0 0x7fc624152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc623fff700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc624157470 0x7fc62413c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc6237fe700 1 -- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6241570a0 msgr2=0x7fc624152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc6237fe700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6241570a0 0x7fc624152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc6237fe700 1 -- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc624157470 msgr2=0x7fc62413c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc6237fe700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc624157470 0x7fc62413c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.810+0000 7fc6237fe700 1 -- 172.21.15.7:0/2102052734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c009420 con 0x7fc62406bb60 2023-10-07T08:01:28.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.811+0000 7fc6237fe700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62406bb60 0x7fc6240c5150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc6240cbc20 tx=0x7fc614017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:28.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.811+0000 7fc61b7fe700 1 -- 172.21.15.7:0/2102052734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614011b10 con 0x7fc62406bb60 2023-10-07T08:01:28.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.811+0000 7fc61b7fe700 1 -- 172.21.15.7:0/2102052734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6140233f0 con 0x7fc62406bb60 2023-10-07T08:01:28.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.811+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc614009420 con 0x7fc62406bb60 2023-10-07T08:01:28.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.811+0000 7fc61b7fe700 1 -- 172.21.15.7:0/2102052734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614023770 con 0x7fc62406bb60 2023-10-07T08:01:28.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.811+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc624154e60 con 0x7fc62406bb60 2023-10-07T08:01:28.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.812+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6240716c0 con 0x7fc62406bb60 2023-10-07T08:01:28.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.812+0000 7fc61b7fe700 1 -- 172.21.15.7:0/2102052734 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc61402c3b0 con 0x7fc62406bb60 2023-10-07T08:01:28.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.813+0000 7fc61b7fe700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc604066f30 0x7fc6040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:28.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.813+0000 7fc61b7fe700 1 -- 172.21.15.7:0/2102052734 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc614085720 con 0x7fc62406bb60 2023-10-07T08:01:28.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.813+0000 7fc622ffd700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc604066f30 0x7fc6040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:28.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.813+0000 7fc622ffd700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc604066f30 0x7fc6040693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc624139c10 tx=0x7fc60c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:28.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:28.817+0000 7fc61b7fe700 1 -- 172.21.15.7:0/2102052734 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc614052d10 con 0x7fc62406bb60 2023-10-07T08:01:29.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.014+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 --> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc624155310 con 0x7fc604066f30 2023-10-07T08:01:29.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.017+0000 7fc61b7fe700 1 -- 172.21.15.7:0/2102052734 <== mgr.4102 v2:172.21.15.172:6832/119541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50563 (secure 0 0 0) 0x7fc60c0099e0 con 0x7fc604066f30 2023-10-07T08:01:29.019 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:01:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc604066f30 msgr2=0x7fc6040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc604066f30 0x7fc6040693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc624139c10 tx=0x7fc60c008040 comp rx=0 tx=0).stop 2023-10-07T08:01:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62406bb60 msgr2=0x7fc6240c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62406bb60 0x7fc6240c5150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc6240cbc20 tx=0x7fc614017000 comp rx=0 tx=0).stop 2023-10-07T08:01:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 shutdown_connections 2023-10-07T08:01:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fc604066f30 0x7fc6040693e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc624157470 0x7fc62413c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62406bb60 0x7fc6240c5150 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 --2- 172.21.15.7:0/2102052734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6241570a0 0x7fc624152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 >> 172.21.15.7:0/2102052734 conn(0x7fc624067ac0 msgr2=0x7fc624139d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 shutdown_connections 2023-10-07T08:01:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.020+0000 7fc629fc7700 1 -- 172.21.15.7:0/2102052734 wait complete. 2023-10-07T08:01:29.023 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-10-07T08:01:29.034 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2023-10-07T08:01:28.076279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2466344,"kb_used_data":6120,"kb_used_omap":0,"kb_used_meta":2460160,"kb_avail":752508376,"statfs":{"total":773094113280,"available":770568577024,"internally_reserved":0,"allocated":6266880,"data_stored":3278608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2519203840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002740"},"pg_stats":[{"pgid":"1.0","version":"53'119","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-07T08:01:05.594875+0000","last_change":"2023-10-07T08:01:05.594838+0000","last_active":"2023-10-07T08:01:05.594875+0000","last_peered":"2023-10-07T08:01:05.594875+0000","last_clean":"2023-10-07T08:01:05.594875+0000","last_became_active":"2023-10-07T07:57:45.953033+0000","last_became_peered":"2023-10-07T07:57:45.953033+0000","last_unstale":"2023-10-07T08:01:05.594875+0000","last_undegraded":"2023-10-07T08:01:05.594875+0000","last_fullsized":"2023-10-07T08:01:05.594875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2023-10-07T08:01:05.594704+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2023-10-07T08:01:05.594704+0000","last_clean_scrub_stamp":"2023-10-07T08:01:05.594704+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T10:22:06.106593+0000","scrub_duration":0.0086142990000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306100,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065740,"statfs":{"total":96636764160,"available":96323317760,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63500000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60699999999999998}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.54700000000000004,"15min":0.55500000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63500000000000001,"5min":1.127,"15min":1.127},"last":0.502},{"interface":"front","average":{"1min":0.433,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58299999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76900000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.626,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70599999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68700000000000006}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.437,"15min":0.45400000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60299999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48799999999999999,"15min":0.51400000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59499999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.433}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.50900000000000001,"15min":0.52500000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.53500000000000003,"15min":0.55600000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":1.048,"15min":1.048},"last":0.55600000000000005}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.53400000000000003,"15min":0.55500000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67700000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.57999999999999996,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80300000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.57799999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73799999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.60099999999999998,"15min":0.61599999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57599999999999996}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.75700000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.63600000000000001,"15min":0.68100000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66300000000000003,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.54700000000000004}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.69799999999999995,"15min":0.77200000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69099999999999995,"5min":2.653,"15min":2.653},"last":0.626},{"interface":"front","average":{"1min":0.496,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70999999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66500000000000004,"15min":0.72699999999999998},"min":{"1min":0.314,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67600000000000005,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60599999999999998,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48799999999999999,"15min":0.504},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.625,"5min":0.878,"15min":0.878},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64400000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.58499999999999996,"15min":0.60199999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66500000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66100000000000003}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.502}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310776,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061064,"statfs":{"total":96636764160,"available":96318529536,"internally_reserved":0,"allocated":1228800,"data_stored":852602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.443,"15min":0.46899999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53400000000000003,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.56699999999999995,"15min":0.63800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52600000000000002,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.40300000000000002}]},{"osd":2,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.66400000000000003,"15min":0.77600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78100000000000003,"5min":7.9909999999999997,"15min":7.9909999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.54400000000000004,"15min":0.57899999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.47199999999999998}]},{"osd":3,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71499999999999997,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.433},{"interface":"front","average":{"1min":0.437,"5min":0.59099999999999997,"15min":0.64700000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70799999999999996,"5min":3.5550000000000002,"15min":3.5550000000000002},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65400000000000003,"15min":0.66700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":1.452,"15min":1.452},"last":0.745}]},{"osd":5,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.996,"5min":3.3929999999999998,"15min":3.3929999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81499999999999995,"5min":1.546,"15min":1.546},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.016,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.81299999999999994,"15min":0.87},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96799999999999997,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.65700000000000003}]},{"osd":7,"last update":"Sat Oct 7 08:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.82199999999999995,"15min":0.879},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0309999999999999,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.78500000000000003,"5min":1.585,"15min":1.585},"last":0.76500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.41199999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.70999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.41699999999999998}]},{"osd":2,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42999999999999999,"15min":0.44},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70099999999999996,"5min":1.298,"15min":1.298},"last":0.443},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68100000000000005,"5min":1.073,"15min":1.073},"last":0.52600000000000002}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.71099999999999997,"5min":1.032,"15min":1.032},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.141,"15min":1.141},"last":0.502}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88800000000000001,"5min":1.218,"15min":1.218},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.753}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":0.69099999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82799999999999996,"5min":1.375,"15min":1.375},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86099999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.401,"15min":1.401},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.69499999999999995,"15min":0.72299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90200000000000002,"5min":1.462,"15min":1.462},"last":0.73199999999999998}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83699999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.71799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.40000000000000002,"15min":0.40699999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.39400000000000002,"15min":0.39600000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.45700000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.47799999999999998}]},{"osd":1,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.51100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59599999999999997}]},{"osd":3,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.435},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.75900000000000001}]},{"osd":5,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.026,"5min":1.51,"15min":1.51},"last":0.65200000000000002}]},{"osd":6,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.88,"15min":0.88},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.63100000000000001}]},{"osd":7,"last update":"Sat Oct 7 08:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.68600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.84499999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.68799999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.41699999999999998,"15min":0.437},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.53100000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42599999999999999,"15min":0.435},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64100000000000001,"5min":1.016,"15min":1.016},"last":0.36099999999999999}]},{"osd":1,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.44500000000000001,"15min":0.46600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.53400000000000003,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.624,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.441}]},{"osd":2,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.49399999999999999,"15min":0.51700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67900000000000005,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.60099999999999998,"5min":1.323,"15min":1.323},"last":0.51600000000000001}]},{"osd":4,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93999999999999995,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.503},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96899999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.76200000000000001}]},{"osd":5,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.751,"15min":0.79000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97299999999999998,"5min":1.327,"15min":1.327},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.78800000000000003,"15min":0.83799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74399999999999999}]},{"osd":6,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.751,"15min":0.78700000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.78200000000000003,"15min":0.81999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.018,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Sat Oct 7 08:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.72099999999999997,"15min":0.74299999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.89300000000000002,"5min":1.528,"15min":1.528},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0369999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.626}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310196,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061644,"statfs":{"total":96636764160,"available":96319123456,"internally_reserved":0,"allocated":634880,"data_stored":262234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.754},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.82599999999999996}]},{"osd":1,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.65400000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.74199999999999999}]},{"osd":2,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.80600000000000005}]},{"osd":3,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.77700000000000002}]},{"osd":5,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.48599999999999999,"15min":0.504},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77400000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.54400000000000004}]},{"osd":6,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.56399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":0.878},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Oct 7 08:00:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57999999999999996,"15min":0.59899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80600000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298488,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073352,"statfs":{"total":96636764160,"available":96331112448,"internally_reserved":0,"allocated":1228800,"data_stored":852602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.60999999999999999,"15min":0.65600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.58999999999999997,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.65400000000000003,"15min":0.70699999999999996},"min":{"1min":0.35599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":3.375,"15min":3.375},"last":0.60399999999999998}]},{"osd":1,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.61699999999999999,"15min":0.66700000000000004},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64300000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.442,"5min":0.62,"15min":0.66800000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.71499999999999997,"15min":0.79900000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":5.2699999999999996,"15min":5.2699999999999996},"last":0.72199999999999998}]},{"osd":3,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89900000000000002,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.91500000000000004,"15min":1.0860000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":10.644,"15min":10.644},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.54300000000000004,"15min":0.58299999999999996},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57399999999999995,"5min":1.242,"15min":1.242},"last":0.39200000000000002}]},{"osd":6,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.51700000000000002,"15min":0.53600000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.64000000000000001,"15min":0.68899999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63600000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.46800000000000003}]},{"osd":7,"last update":"Sat Oct 7 08:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.59199999999999997,"15min":0.61399999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.75600000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65100000000000002,"5min":1.123,"15min":1.123},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T08:01:29.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-07T08:01:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.168+0000 7fb0e3309700 1 Processor -- start 2023-10-07T08:01:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.168+0000 7fb0e3309700 1 -- start start 2023-10-07T08:01:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e3309700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc156f30 0x7fb0dc0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e3309700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0dc0b0190 0x7fb0dc0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e3309700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc0b18d0 con 0x7fb0dc0b22a0 2023-10-07T08:01:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e3309700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc0b0fa0 con 0x7fb0dc0b0190 2023-10-07T08:01:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e3309700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0b1280 con 0x7fb0dc156f30 2023-10-07T08:01:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e18a6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0dc0b0190 0x7fb0dc0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e10a5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc156f30 0x7fb0dc0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e08a4700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb0dc0b22a0 0x7fb0dc0afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.7:58258/0 (socket says 172.21.15.7:58258) 2023-10-07T08:01:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e18a6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0dc0b0190 0x7fb0dc0b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.7:60590/0 (socket says 172.21.15.7:60590) 2023-10-07T08:01:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e08a4700 1 -- 172.21.15.7:0/1639626600 learned_addr learned my addr 172.21.15.7:0/1639626600 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e10a5700 1 -- 172.21.15.7:0/1639626600 >> v2:172.21.15.172:3300/0 conn(0x7fb0dc0b0190 msgr2=0x7fb0dc0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e10a5700 1 --2- 172.21.15.7:0/1639626600 >> v2:172.21.15.172:3300/0 conn(0x7fb0dc0b0190 0x7fb0dc0b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.169+0000 7fb0e10a5700 1 -- 172.21.15.7:0/1639626600 >> v1:172.21.15.172:6789/0 conn(0x7fb0dc0b22a0 legacy=0x7fb0dc0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.170+0000 7fb0d3fff700 1 -- 172.21.15.7:0/1639626600 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430600445 0 0) 0x7fb0dc0b18d0 con 0x7fb0dc0b22a0 2023-10-07T08:01:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.170+0000 7fb0e10a5700 1 -- 172.21.15.7:0/1639626600 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0b1560 con 0x7fb0dc156f30 2023-10-07T08:01:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.170+0000 7fb0e18a6700 1 --2- 172.21.15.7:0/1639626600 >> v2:172.21.15.172:3300/0 conn(0x7fb0dc0b0190 0x7fb0dc0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.170+0000 7fb0e10a5700 1 --2- 172.21.15.7:0/1639626600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc156f30 0x7fb0dc0b1d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80098e0 tx=0x7fb0d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b820bfbbfbb31c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.170+0000 7fb0d3fff700 1 -- 172.21.15.7:0/1639626600 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800b640 con 0x7fb0dc156f30 2023-10-07T08:01:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.170+0000 7fb0d3fff700 1 -- 172.21.15.7:0/1639626600 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d800b7a0 con 0x7fb0dc156f30 2023-10-07T08:01:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.170+0000 7fb0d3fff700 1 -- 172.21.15.7:0/1639626600 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800bb40 con 0x7fb0dc156f30 2023-10-07T08:01:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.170+0000 7fb0e3309700 1 -- 172.21.15.7:0/1639626600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc156f30 msgr2=0x7fb0dc0b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.170+0000 7fb0e3309700 1 --2- 172.21.15.7:0/1639626600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc156f30 0x7fb0dc0b1d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80098e0 tx=0x7fb0d80091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.171+0000 7fb0e3309700 1 -- 172.21.15.7:0/1639626600 shutdown_connections 2023-10-07T08:01:29.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.171+0000 7fb0e3309700 1 --2- 172.21.15.7:0/1639626600 >> v2:172.21.15.172:3300/0 conn(0x7fb0dc0b0190 0x7fb0dc0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.171+0000 7fb0e3309700 1 --2- 172.21.15.7:0/1639626600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc156f30 0x7fb0dc0b1d60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.171+0000 7fb0e3309700 1 -- 172.21.15.7:0/1639626600 >> 172.21.15.7:0/1639626600 conn(0x7fb0dc06d350 msgr2=0x7fb0dc06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:29.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.171+0000 7fb0e3309700 1 -- 172.21.15.7:0/1639626600 shutdown_connections 2023-10-07T08:01:29.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.171+0000 7fb0e3309700 1 -- 172.21.15.7:0/1639626600 wait complete. 2023-10-07T08:01:29.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.172+0000 7fb0e3309700 1 Processor -- start 2023-10-07T08:01:29.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.172+0000 7fb0e3309700 1 -- start start 2023-10-07T08:01:29.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.173+0000 7fb0e3309700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc0b0190 0x7fb0dc151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.173+0000 7fb0e3309700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0dc0b22a0 0x7fb0dc1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e3309700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc156f30 0x7fb0dc156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e3309700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc151fb0 con 0x7fb0dc156f30 2023-10-07T08:01:29.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e3309700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc154e20 con 0x7fb0dc0b22a0 2023-10-07T08:01:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e3309700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1550e0 con 0x7fb0dc0b0190 2023-10-07T08:01:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e18a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc156f30 0x7fb0dc156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e18a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc156f30 0x7fb0dc156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55974/0 (socket says 172.21.15.7:55974) 2023-10-07T08:01:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e18a6700 1 -- 172.21.15.7:0/130756688 learned_addr learned my addr 172.21.15.7:0/130756688 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e10a5700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc0b0190 0x7fb0dc151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e08a4700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0dc0b22a0 0x7fb0dc1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e18a6700 1 -- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc0b0190 msgr2=0x7fb0dc151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e18a6700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc0b0190 0x7fb0dc151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e18a6700 1 -- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0dc0b22a0 msgr2=0x7fb0dc1548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e18a6700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0dc0b22a0 0x7fb0dc1548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e18a6700 1 -- 172.21.15.7:0/130756688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d400de30 con 0x7fb0dc156f30 2023-10-07T08:01:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e08a4700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0dc0b22a0 0x7fb0dc1548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T08:01:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e18a6700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc156f30 0x7fb0dc156180 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40124f0 tx=0x7fb0d400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0d17fa700 1 -- 172.21.15.7:0/130756688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4012e50 con 0x7fb0dc156f30 2023-10-07T08:01:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0d17fa700 1 -- 172.21.15.7:0/130756688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d4013040 con 0x7fb0dc156f30 2023-10-07T08:01:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d8009420 con 0x7fb0dc156f30 2023-10-07T08:01:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.174+0000 7fb0d17fa700 1 -- 172.21.15.7:0/130756688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4019830 con 0x7fb0dc156f30 2023-10-07T08:01:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.175+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc1556a0 con 0x7fb0dc156f30 2023-10-07T08:01:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.176+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc04c3c0 con 0x7fb0dc156f30 2023-10-07T08:01:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.176+0000 7fb0d17fa700 1 -- 172.21.15.7:0/130756688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0d4019be0 con 0x7fb0dc156f30 2023-10-07T08:01:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.176+0000 7fb0d17fa700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb0c0066f30 0x7fb0c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.176+0000 7fb0d17fa700 1 -- 172.21.15.7:0/130756688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb0d4089380 con 0x7fb0dc156f30 2023-10-07T08:01:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.177+0000 7fb0e10a5700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb0c0066f30 0x7fb0c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.177+0000 7fb0e10a5700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb0c0066f30 0x7fb0c00693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80091e0 tx=0x7fb0d8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.181+0000 7fb0d17fa700 1 -- 172.21.15.7:0/130756688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0d40568f0 con 0x7fb0dc156f30 2023-10-07T08:01:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.386+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fb0dc0b3d50 con 0x7fb0dc156f30 2023-10-07T08:01:29.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.389+0000 7fb0d17fa700 1 -- 172.21.15.7:0/130756688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb0d4019e90 con 0x7fb0dc156f30 2023-10-07T08:01:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.390+0000 7fb0d17fa700 1 -- 172.21.15.7:0/130756688 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7fb0d4059f10 con 0x7fb0dc156f30 2023-10-07T08:01:29.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb0c0066f30 msgr2=0x7fb0c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb0c0066f30 0x7fb0c00693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80091e0 tx=0x7fb0d8017000 comp rx=0 tx=0).stop 2023-10-07T08:01:29.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc156f30 msgr2=0x7fb0dc156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc156f30 0x7fb0dc156180 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40124f0 tx=0x7fb0d400db90 comp rx=0 tx=0).stop 2023-10-07T08:01:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 shutdown_connections 2023-10-07T08:01:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7fb0c0066f30 0x7fb0c00693e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc156f30 0x7fb0dc156180 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0dc0b0190 0x7fb0dc151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 --2- 172.21.15.7:0/130756688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0dc0b22a0 0x7fb0dc1548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 >> 172.21.15.7:0/130756688 conn(0x7fb0dc06d350 msgr2=0x7fb0dc152900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 shutdown_connections 2023-10-07T08:01:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.393+0000 7fb0e3309700 1 -- 172.21.15.7:0/130756688 wait complete. 2023-10-07T08:01:29.408 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-07T08:01:29.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T08:01:29.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.540+0000 7f4770f3d700 1 Processor -- start 2023-10-07T08:01:29.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.540+0000 7f4770f3d700 1 -- start start 2023-10-07T08:01:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f4770f3d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f476c159690 0x7f476c0cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f4770f3d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 0x7f476c0a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f4770f3d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c06bb70 con 0x7f476c1592c0 2023-10-07T08:01:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f4770f3d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476c0cda10 con 0x7f476c159690 2023-10-07T08:01:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f4770f3d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c06af60 con 0x7f476c0cb0c0 2023-10-07T08:01:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f476a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f476c1592c0 0x7f476c06a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45686/0 (socket says 172.21.15.7:45686) 2023-10-07T08:01:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f476a59c700 1 -- 172.21.15.7:0/1303967255 learned_addr learned my addr 172.21.15.7:0/1303967255 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-10-07T08:01:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f4769d9b700 1 --2- 172.21.15.7:0/1303967255 >> v2:172.21.15.172:3300/0 conn(0x7f476c159690 0x7f476c0cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f476ad9d700 1 --2- 172.21.15.7:0/1303967255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 0x7f476c0a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f476959a700 1 -- 172.21.15.7:0/1303967255 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218505527 0 0) 0x7f476c06bb70 con 0x7f476c1592c0 2023-10-07T08:01:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.541+0000 7f476959a700 1 -- 172.21.15.7:0/1303967255 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4758003570 con 0x7f476c1592c0 2023-10-07T08:01:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f4769d9b700 1 -- 172.21.15.7:0/1303967255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 msgr2=0x7f476c0a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T08:01:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f4769d9b700 1 --2- 172.21.15.7:0/1303967255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 0x7f476c0a9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f476959a700 1 -- 172.21.15.7:0/1303967255 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1725078835 0 0) 0x7f4758003570 con 0x7f476c1592c0 2023-10-07T08:01:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f4769d9b700 1 -- 172.21.15.7:0/1303967255 >> v1:172.21.15.7:6789/0 conn(0x7f476c1592c0 legacy=0x7f476c06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f4769d9b700 1 -- 172.21.15.7:0/1303967255 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c06b240 con 0x7f476c159690 2023-10-07T08:01:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f476ad9d700 1 --2- 172.21.15.7:0/1303967255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 0x7f476c0a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T08:01:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f4769d9b700 1 --2- 172.21.15.7:0/1303967255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f476c159690 0x7f476c0cab80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f475c0098e0 tx=0x7f475c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8d9e4f0ef27520f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f476959a700 1 -- 172.21.15.7:0/1303967255 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754001470 con 0x7f476c159690 2023-10-07T08:01:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f476959a700 1 -- 172.21.15.7:0/1303967255 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f475c011420 con 0x7f476c159690 2023-10-07T08:01:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.542+0000 7f476959a700 1 -- 172.21.15.7:0/1303967255 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0117e0 con 0x7f476c159690 2023-10-07T08:01:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.543+0000 7f4770f3d700 1 -- 172.21.15.7:0/1303967255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f476c159690 msgr2=0x7f476c0cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.543+0000 7f4770f3d700 1 --2- 172.21.15.7:0/1303967255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f476c159690 0x7f476c0cab80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f475c0098e0 tx=0x7f475c0091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.543+0000 7f4770f3d700 1 -- 172.21.15.7:0/1303967255 shutdown_connections 2023-10-07T08:01:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.543+0000 7f4770f3d700 1 --2- 172.21.15.7:0/1303967255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 0x7f476c0a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.543+0000 7f4770f3d700 1 --2- 172.21.15.7:0/1303967255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f476c159690 0x7f476c0cab80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.543+0000 7f4770f3d700 1 -- 172.21.15.7:0/1303967255 >> 172.21.15.7:0/1303967255 conn(0x7f476c068bd0 msgr2=0x7f476c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.543+0000 7f4770f3d700 1 -- 172.21.15.7:0/1303967255 shutdown_connections 2023-10-07T08:01:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.543+0000 7f4770f3d700 1 -- 172.21.15.7:0/1303967255 wait complete. 2023-10-07T08:01:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.544+0000 7f4770f3d700 1 Processor -- start 2023-10-07T08:01:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.544+0000 7f4770f3d700 1 -- start start 2023-10-07T08:01:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4770f3d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 0x7f476c15e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4770f3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1592c0 0x7f476c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4770f3d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f476c159690 0x7f476c0c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4770f3d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0ac700 con 0x7f476c1592c0 2023-10-07T08:01:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4770f3d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c15ed20 con 0x7f476c159690 2023-10-07T08:01:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4770f3d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c169e80 con 0x7f476c0cb0c0 2023-10-07T08:01:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4769d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1592c0 0x7f476c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4769d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1592c0 0x7f476c15d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39920/0 (socket says 172.21.15.7:39920) 2023-10-07T08:01:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4769d9b700 1 -- 172.21.15.7:0/3391375089 learned_addr learned my addr 172.21.15.7:0/3391375089 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-10-07T08:01:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f476ad9d700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f476c159690 0x7f476c0c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f476a59c700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 0x7f476c15e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4769d9b700 1 -- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 msgr2=0x7f476c15e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4769d9b700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 0x7f476c15e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4769d9b700 1 -- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f476c159690 msgr2=0x7f476c0c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4769d9b700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f476c159690 0x7f476c0c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4769d9b700 1 -- 172.21.15.7:0/3391375089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764009a20 con 0x7f476c1592c0 2023-10-07T08:01:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.545+0000 7f4769d9b700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1592c0 0x7f476c15d980 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f475c0030c0 tx=0x7f475c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.546+0000 7f47627fc700 1 -- 172.21.15.7:0/3391375089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c01cba0 con 0x7f476c1592c0 2023-10-07T08:01:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.546+0000 7f47627fc700 1 -- 172.21.15.7:0/3391375089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f475c01cd20 con 0x7f476c1592c0 2023-10-07T08:01:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.546+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c009420 con 0x7f476c1592c0 2023-10-07T08:01:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.546+0000 7f47627fc700 1 -- 172.21.15.7:0/3391375089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c01a6c0 con 0x7f476c1592c0 2023-10-07T08:01:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.546+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c16a440 con 0x7f476c1592c0 2023-10-07T08:01:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.547+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c0ab550 con 0x7f476c1592c0 2023-10-07T08:01:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.547+0000 7f47627fc700 1 -- 172.21.15.7:0/3391375089 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f475c01aaf0 con 0x7f476c1592c0 2023-10-07T08:01:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.548+0000 7f47627fc700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f474c066f30 0x7f474c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T08:01:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.548+0000 7f47627fc700 1 -- 172.21.15.7:0/3391375089 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f475c0859e0 con 0x7f476c1592c0 2023-10-07T08:01:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.548+0000 7f476a59c700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f474c066f30 0x7f474c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T08:01:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.548+0000 7f476a59c700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f474c066f30 0x7f474c0693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f476c0b1e50 tx=0x7f4754001aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T08:01:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.552+0000 7f47627fc700 1 -- 172.21.15.7:0/3391375089 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f475c053110 con 0x7f476c1592c0 2023-10-07T08:01:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.803+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f476c0716c0 con 0x7f476c1592c0 2023-10-07T08:01:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.804+0000 7f47627fc700 1 -- 172.21.15.7:0/3391375089 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f475c029090 con 0x7f476c1592c0 2023-10-07T08:01:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-10-07T08:01:29.805 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3622385241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3622385241},{"type":"v1","addr":"172.21.15.172:6837","nonce":3622385241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/899764672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":899764672},{"type":"v1","addr":"172.21.15.172:6836","nonce":899764672}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/318366060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":318366060},{"type":"v1","addr":"172.21.15.7:6833","nonce":318366060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/917828347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":917828347},{"type":"v1","addr":"172.21.15.7:6835","nonce":917828347}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[]} 2023-10-07T08:01:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.806+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f474c066f30 msgr2=0x7f474c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.806+0000 7f4770f3d700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f474c066f30 0x7f474c0693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f476c0b1e50 tx=0x7f4754001aa0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.806+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1592c0 msgr2=0x7f476c15d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T08:01:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.806+0000 7f4770f3d700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1592c0 0x7f476c15d980 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f475c0030c0 tx=0x7f475c0091a0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.806+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 shutdown_connections 2023-10-07T08:01:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.806+0000 7f4770f3d700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:6832/119541,v1:172.21.15.172:6833/119541] conn(0x7f474c066f30 0x7f474c0693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.806+0000 7f4770f3d700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f476c159690 0x7f476c0c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.806+0000 7f4770f3d700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f476c0cb0c0 0x7f476c15e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.806+0000 7f4770f3d700 1 --2- 172.21.15.7:0/3391375089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1592c0 0x7f476c15d980 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T08:01:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.807+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 >> 172.21.15.7:0/3391375089 conn(0x7f476c068bd0 msgr2=0x7f476c06f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T08:01:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.807+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 shutdown_connections 2023-10-07T08:01:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-10-07T08:01:29.807+0000 7f4770f3d700 1 -- 172.21.15.7:0/3391375089 wait complete. 2023-10-07T08:01:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2023-10-07T08:01:29.822 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-07T08:01:29.822 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-07T08:01:29.825 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-07T08:01:29.826 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:29.835 INFO:tasks.ceph.mds.a:Stopped 2023-10-07T08:01:29.835 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-07T08:01:29.836 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:29.886 INFO:tasks.ceph.mds.c:Stopped 2023-10-07T08:01:29.887 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-07T08:01:29.887 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:29.898 INFO:tasks.ceph.mds.b:Stopped 2023-10-07T08:01:29.898 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-07T08:01:29.898 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:29.949 INFO:tasks.ceph.mds.d:Stopped 2023-10-07T08:01:29.949 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-07T08:01:29.949 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-07T08:01:29.950 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:29.992 INFO:tasks.ceph.osd.0:Stopped 2023-10-07T08:01:29.993 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-07T08:01:29.993 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.029 INFO:tasks.ceph.osd.1:Stopped 2023-10-07T08:01:30.030 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-07T08:01:30.030 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.100 INFO:tasks.ceph.osd.2:Stopped 2023-10-07T08:01:30.101 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-07T08:01:30.101 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.172 INFO:tasks.ceph.osd.3:Stopped 2023-10-07T08:01:30.172 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-07T08:01:30.173 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.213 INFO:tasks.ceph.osd.4:Stopped 2023-10-07T08:01:30.213 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-07T08:01:30.213 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.285 INFO:tasks.ceph.osd.5:Stopped 2023-10-07T08:01:30.286 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-07T08:01:30.286 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.356 INFO:tasks.ceph.osd.6:Stopped 2023-10-07T08:01:30.356 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-07T08:01:30.356 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.422 INFO:tasks.ceph.osd.7:Stopped 2023-10-07T08:01:30.423 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-07T08:01:30.423 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-07T08:01:30.423 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.449 INFO:tasks.ceph.mgr.y:Stopped 2023-10-07T08:01:30.450 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-07T08:01:30.450 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.488 INFO:tasks.ceph.mgr.x:Stopped 2023-10-07T08:01:30.489 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-07T08:01:30.489 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-07T08:01:30.489 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.503 INFO:tasks.ceph.mon.a:Stopped 2023-10-07T08:01:30.504 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-07T08:01:30.504 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.540 INFO:tasks.ceph.mon.b:Stopped 2023-10-07T08:01:30.540 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-07T08:01:30.541 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T08:01:30.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-10-07T08:01:30.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-10-07T08:01:30.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-10-07T08:01:30.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-10-07T08:01:30.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-10-07T08:01:30.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-10-07T08:01:30.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-10-07T08:01:30.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-10-07T08:01:30.593 INFO:tasks.ceph.mon.c:Stopped 2023-10-07T08:01:30.593 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-07T08:01:30.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2023-10-07T08:01:30.642 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T08:01:30.642 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-07T08:01:30.846 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T08:01:30.846 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-07T08:01:30.949 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T08:01:30.950 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-07T08:01:31.032 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T08:01:31.033 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-07T08:01:31.111 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T08:01:31.112 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-07T08:01:31.324 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T08:01:31.324 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-07T08:01:31.415 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T08:01:31.416 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-07T08:01:31.487 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T08:01:31.487 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-07T08:01:31.559 INFO:tasks.ceph:Archiving mon data... 2023-10-07T08:01:31.560 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/data/mon.a.tgz 2023-10-07T08:01:31.560 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2023-10-07T08:01:31.585 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.vGsgNLsB1O 2023-10-07T08:01:31.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vGsgNLsB1O 2023-10-07T08:01:31.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.vGsgNLsB1O 2023-10-07T08:01:31.834 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.vGsgNLsB1O is 326KB 2023-10-07T08:01:31.931 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.vGsgNLsB1O 2023-10-07T08:01:31.956 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/data/mon.b.tgz 2023-10-07T08:01:31.957 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2023-10-07T08:01:31.979 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.AtwU0hpa1v 2023-10-07T08:01:31.979 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.AtwU0hpa1v 2023-10-07T08:01:32.146 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.AtwU0hpa1v 2023-10-07T08:01:32.236 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.AtwU0hpa1v is 430KB 2023-10-07T08:01:32.333 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.AtwU0hpa1v 2023-10-07T08:01:32.352 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/data/mon.c.tgz 2023-10-07T08:01:32.352 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2023-10-07T08:01:32.408 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.wfnVThovaU 2023-10-07T08:01:32.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.wfnVThovaU 2023-10-07T08:01:32.553 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.wfnVThovaU 2023-10-07T08:01:32.641 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.wfnVThovaU is 430KB 2023-10-07T08:01:32.741 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.wfnVThovaU 2023-10-07T08:01:32.759 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-07T08:01:32.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-07T08:01:32.762 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-07T08:01:32.826 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-07T08:01:32.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-07T08:01:32.837 DEBUG:teuthology.orchestra.run.smithi172:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-07T08:01:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-07T08:01:32.894 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-07T08:01:32.900 INFO:tasks.ceph:Archiving crash dumps... 2023-10-07T08:01:32.901 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/remote/smithi007/crash 2023-10-07T08:01:32.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-07T08:01:32.954 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/remote/smithi172/crash 2023-10-07T08:01:32.955 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-07T08:01:32.982 INFO:tasks.ceph:Shutting down logrotate 2023-10-07T08:01:32.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T08:01:32.999 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T08:01:33.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-07T08:01:33.115 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-07T08:01:33.142 INFO:tasks.ceph:Compressing logs... 2023-10-07T08:01:33.143 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-07T08:01:33.158 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-07T08:01:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131717.log 2023-10-07T08:01:33.192 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127167.log 2023-10-07T08:01:33.192 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129981.log 2023-10-07T08:01:33.192 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131717.log.gz 2023-10-07T08:01:33.192 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127094.log 2023-10-07T08:01:33.192 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127167.log.gz 2023-10-07T08:01:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120130.log 2023-10-07T08:01:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.129981.log: /var/log/ceph/ceph-client.admin.127094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138953.log 2023-10-07T08:01:33.196 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129981.log.gz 2023-10-07T08:01:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.120130.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127094.log.gz 2023-10-07T08:01:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136019.log 2023-10-07T08:01:33.196 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.120130.log.gz 2023-10-07T08:01:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131036.log 2023-10-07T08:01:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.136019.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138953.log.gz 2023-10-07T08:01:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130325.log 2023-10-07T08:01:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131036.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136019.log.gz 2023-10-07T08:01:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140427.log 2023-10-07T08:01:33.197 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131036.log.gz 2023-10-07T08:01:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127676.log 2023-10-07T08:01:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.140427.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.130325.log.gz 2023-10-07T08:01:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126748.log 2023-10-07T08:01:33.198 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140427.log.gz 2023-10-07T08:01:33.198 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132296.log 2023-10-07T08:01:33.198 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127676.log.gz 2023-10-07T08:01:33.198 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131519.log 2023-10-07T08:01:33.198 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132296.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126748.log.gz 2023-10-07T08:01:33.198 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132800.log 2023-10-07T08:01:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131519.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.132296.log.gz 2023-10-07T08:01:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132051.log 2023-10-07T08:01:33.207 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131519.log.gz 2023-10-07T08:01:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135426.log 2023-10-07T08:01:33.207 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.132800.log.gz 2023-10-07T08:01:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138273.log 2023-10-07T08:01:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135426.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.132051.log.gz 2023-10-07T08:01:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134595.log 2023-10-07T08:01:33.208 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135426.log.gz 2023-10-07T08:01:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136068.log 2023-10-07T08:01:33.208 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.138273.log.gz 2023-10-07T08:01:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133020.log 2023-10-07T08:01:33.208 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.134595.log.gz/var/log/ceph/ceph-client.admin.136068.log: 2023-10-07T08:01:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136117.log 2023-10-07T08:01:33.209 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136068.log.gz 2023-10-07T08:01:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.133020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128019.log 2023-10-07T08:01:33.209 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.133020.log.gz 2023-10-07T08:01:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.136117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125384.log 2023-10-07T08:01:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.128019.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136117.log.gz 2023-10-07T08:01:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130178.log 2023-10-07T08:01:33.209 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128019.log.gz 2023-10-07T08:01:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138733.log 2023-10-07T08:01:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125384.log: /var/log/ceph/ceph-client.admin.130178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139256.log 2023-10-07T08:01:33.210 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.125384.log.gz 2023-10-07T08:01:33.210 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130178.log.gz 2023-10-07T08:01:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132702.log 2023-10-07T08:01:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139256.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138733.log.gz 2023-10-07T08:01:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125690.log 2023-10-07T08:01:33.210 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139256.log.gz 2023-10-07T08:01:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139857.log 2023-10-07T08:01:33.211 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.132702.log.gz 2023-10-07T08:01:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125533.log 2023-10-07T08:01:33.211 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125690.log.gz 2023-10-07T08:01:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132604.log 2023-10-07T08:01:33.211 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.139857.log.gz 2023-10-07T08:01:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130619.log 2023-10-07T08:01:33.211 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125533.log.gz 2023-10-07T08:01:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131403.log 2023-10-07T08:01:33.212 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.132604.log.gz 2023-10-07T08:01:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131806.log 2023-10-07T08:01:33.212 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130619.log.gz 2023-10-07T08:01:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130129.log 2023-10-07T08:01:33.212 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131403.log.gz 2023-10-07T08:01:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125790.log 2023-10-07T08:01:33.212 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131806.log.gz 2023-10-07T08:01:33.213 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133577.log 2023-10-07T08:01:33.213 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130129.log.gz 2023-10-07T08:01:33.213 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134644.log 2023-10-07T08:01:33.213 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.133577.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125790.log.gz 2023-10-07T08:01:33.213 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127627.log 2023-10-07T08:01:33.213 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133577.log.gz 2023-10-07T08:01:33.213 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130521.log 2023-10-07T08:01:33.213 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134644.log.gz 2023-10-07T08:01:33.214 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125888.log 2023-10-07T08:01:33.214 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127627.log.gz 2023-10-07T08:01:33.214 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130521.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-07T08:01:33.214 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130521.log.gz 2023-10-07T08:01:33.214 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134791.log 2023-10-07T08:01:33.214 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125888.log.gz 2023-10-07T08:01:33.214 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136264.log 2023-10-07T08:01:33.215 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134791.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-07T08:01:33.215 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.134791.log.gz 2023-10-07T08:01:33.215 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.136264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140378.log 2023-10-07T08:01:33.225 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-07T08:01:33.225 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119919.log 2023-10-07T08:01:33.225 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.128298.log 2023-10-07T08:01:33.225 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.119919.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-07T08:01:33.226 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119919.log.gz 2023-10-07T08:01:33.226 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120019.log 2023-10-07T08:01:33.226 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.128298.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.126823.log 2023-10-07T08:01:33.226 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.120019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119772.log 2023-10-07T08:01:33.226 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120019.log.gz 2023-10-07T08:01:33.226 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.126823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.127498.log 2023-10-07T08:01:33.226 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.119772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119820.log 2023-10-07T08:01:33.227 INFO:teuthology.orchestra.run.smithi172.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.1.128298.log.gz 2023-10-07T08:01:33.227 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119772.log.gz 2023-10-07T08:01:33.227 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.3.127498.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-07T08:01:33.227 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.1.126823.log.gz 2023-10-07T08:01:33.227 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.119820.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-07T08:01:33.227 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119820.log.gz 2023-10-07T08:01:33.227 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119869.log 2023-10-07T08:01:33.227 INFO:teuthology.orchestra.run.smithi172.stderr: 93.3% 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz -- replaced with /var/log/ceph/ceph-client.3.127498.log.gz 2023-10-07T08:01:33.228 INFO:teuthology.orchestra.run.smithi172.stderr: 2023-10-07T08:01:33.228 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-07T08:01:33.228 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 /var/log/ceph/ceph-client.admin.119869.log: --verbose -- /var/log/ceph/ceph-client.admin.120068.log 2023-10-07T08:01:33.228 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mon.b.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119869.log.gz 2023-10-07T08:01:33.228 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.120068.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.120068.log.gz 2023-10-07T08:01:33.228 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-07T08:01:33.229 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-07T08:01:33.229 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-07T08:01:33.229 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mds.d.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-07T08:01:33.229 INFO:teuthology.orchestra.run.smithi172.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-07T08:01:33.230 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-07T08:01:33.230 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-07T08:01:33.230 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-osd.4.log: 91.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-07T08:01:33.230 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.129102.log 2023-10-07T08:01:33.230 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.129102.log: 93.1% -- replaced with /var/log/ceph/ceph-client.1.129102.log.gz 2023-10-07T08:01:33.230 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-07T08:01:33.233 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119970.log 2023-10-07T08:01:33.233 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.119970.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.119970.log.gz 2023-10-07T08:01:33.233 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120116.log 2023-10-07T08:01:33.234 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.120116.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.120116.log.gz 2023-10-07T08:01:33.256 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136264.log.gz 2023-10-07T08:01:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134546.log 2023-10-07T08:01:33.257 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.140378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136215.log 2023-10-07T08:01:33.257 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140378.log.gz 2023-10-07T08:01:33.258 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137721.log 2023-10-07T08:01:33.258 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134546.log.gz 2023-10-07T08:01:33.258 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.136215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126350.log 2023-10-07T08:01:33.258 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136215.log.gz 2023-10-07T08:01:33.258 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.137721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135562.log 2023-10-07T08:01:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126350.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137721.log.gz 2023-10-07T08:01:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126363.log 2023-10-07T08:01:33.259 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126350.log.gz 2023-10-07T08:01:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132198.log 2023-10-07T08:01:33.259 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135562.log.gz 2023-10-07T08:01:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127823.log 2023-10-07T08:01:33.260 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126363.log.gz 2023-10-07T08:01:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119662.log 2023-10-07T08:01:33.260 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.132198.log.gz 2023-10-07T08:01:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131134.log 2023-10-07T08:01:33.260 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127823.log.gz 2023-10-07T08:01:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.119662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125986.log 2023-10-07T08:01:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131134.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.119662.log.gz 2023-10-07T08:01:33.261 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126733.log 2023-10-07T08:01:33.261 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131134.log.gz 2023-10-07T08:01:33.261 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139710.log 2023-10-07T08:01:33.261 INFO:teuthology.orchestra.run.smithi007.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 87.2% 2023-10-07T08:01:33.261 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.125986.log.gz 2023-10-07T08:01:33.261 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138616.log 2023-10-07T08:01:33.261 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126733.log.gz 2023-10-07T08:01:33.262 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138371.log 2023-10-07T08:01:33.262 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139710.log.gz 2023-10-07T08:01:33.262 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-07T08:01:33.262 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138616.log: /var/log/ceph/ceph-client.admin.138371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130031.log 2023-10-07T08:01:33.262 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.138616.log.gz 2023-10-07T08:01:33.263 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138371.log.gz 2023-10-07T08:01:33.263 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125106.log 2023-10-07T08:01:33.263 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139661.log 2023-10-07T08:01:33.263 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130031.log.gz 2023-10-07T08:01:33.263 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126364.log 2023-10-07T08:01:33.263 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125106.log.gz 2023-10-07T08:01:33.264 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137016.log 2023-10-07T08:01:33.264 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139661.log: 88.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-07T08:01:33.264 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.139661.log.gz 2023-10-07T08:01:33.264 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130938.log 2023-10-07T08:01:33.264 INFO:teuthology.orchestra.run.smithi007.stderr:gzip /var/log/ceph/ceph-client.admin.137016.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.135611.log 2023-10-07T08:01:33.264 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126364.log.gz 2023-10-07T08:01:33.265 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128238.log 2023-10-07T08:01:33.265 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137016.log.gz 2023-10-07T08:01:33.266 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135611.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130938.log.gz 2023-10-07T08:01:33.266 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135919.log 2023-10-07T08:01:33.266 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135611.log.gz 2023-10-07T08:01:33.266 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.128238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135513.log 2023-10-07T08:01:33.267 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128238.log.gz 2023-10-07T08:01:33.267 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130227.log 2023-10-07T08:01:33.267 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135919.log.gz 2023-10-07T08:01:33.267 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140151.log 2023-10-07T08:01:33.268 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130227.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135513.log.gz 2023-10-07T08:01:33.268 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130423.log 2023-10-07T08:01:33.268 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130227.log.gz 2023-10-07T08:01:33.268 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.140151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126133.log 2023-10-07T08:01:33.269 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130423.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140151.log.gz 2023-10-07T08:01:33.269 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130570.log 2023-10-07T08:01:33.269 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130423.log.gz 2023-10-07T08:01:33.269 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130276.log 2023-10-07T08:01:33.269 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126133.log.gz 2023-10-07T08:01:33.269 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130570.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-07T08:01:33.270 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130570.log.gz 2023-10-07T08:01:33.270 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131281.log 2023-10-07T08:01:33.270 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130276.log.gz 2023-10-07T08:01:33.270 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129932.log 2023-10-07T08:01:33.270 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138126.log 2023-10-07T08:01:33.270 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.129932.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131281.log.gz 2023-10-07T08:01:33.271 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125263.log 2023-10-07T08:01:33.271 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129932.log.gz 2023-10-07T08:01:33.271 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138831.log 2023-10-07T08:01:33.271 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138126.log: /var/log/ceph/ceph-client.admin.125263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138322.log 2023-10-07T08:01:33.271 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.138126.log.gz 2023-10-07T08:01:33.271 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138831.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.125263.log.gz 2023-10-07T08:01:33.272 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138567.log 2023-10-07T08:01:33.272 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138831.log.gz 2023-10-07T08:01:33.272 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126355.log 2023-10-07T08:01:33.272 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.138322.log.gz 2023-10-07T08:01:33.272 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125194.log 2023-10-07T08:01:33.272 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138567.log: /var/log/ceph/ceph-client.admin.126355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139612.log 2023-10-07T08:01:33.272 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138567.log.gz 2023-10-07T08:01:33.273 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125194.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126355.log.gz 2023-10-07T08:01:33.273 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130080.log 2023-10-07T08:01:33.273 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.125194.log.gz 2023-10-07T08:01:33.273 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139612.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-07T08:01:33.273 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139612.log.gz 2023-10-07T08:01:33.273 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139955.log 2023-10-07T08:01:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130080.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137905.log 2023-10-07T08:01:33.274 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.130080.log.gz 2023-10-07T08:01:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139563.log 2023-10-07T08:01:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.137905.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.139955.log.gz 2023-10-07T08:01:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135104.log 2023-10-07T08:01:33.274 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137905.log.gz 2023-10-07T08:01:33.275 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125482.log 2023-10-07T08:01:33.275 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135104.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.139563.log.gz 2023-10-07T08:01:33.275 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135104.log.gz 2023-10-07T08:01:33.275 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125482.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125482.log.gz 2023-10-07T08:01:33.275 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126750.log 2023-10-07T08:01:33.275 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126358.log 2023-10-07T08:01:33.275 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139514.log 2023-10-07T08:01:33.276 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126750.log.gz 2023-10-07T08:01:33.276 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129018.log 2023-10-07T08:01:33.276 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133675.log 2023-10-07T08:01:33.276 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139514.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126358.log.gz 2023-10-07T08:01:33.276 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.129018.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.139514.log.gz 2023-10-07T08:01:33.276 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119999.log 2023-10-07T08:01:33.277 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129018.log.gz 2023-10-07T08:01:33.277 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.133675.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.133675.log.gz 2023-10-07T08:01:33.277 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.119999.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.119999.log.gz 2023-10-07T08:01:33.277 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133626.log 2023-10-07T08:01:33.277 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134301.log 2023-10-07T08:01:33.277 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.133626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129791.log 2023-10-07T08:01:33.278 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.133626.log.gz 2023-10-07T08:01:33.278 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127774.log 2023-10-07T08:01:33.278 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.134301.log.gz 2023-10-07T08:01:33.278 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.129791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130987.log 2023-10-07T08:01:33.278 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129791.log.gz 2023-10-07T08:01:33.278 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126756.log 2023-10-07T08:01:33.279 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127774.log.gz 2023-10-07T08:01:33.279 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135153.log 2023-10-07T08:01:33.279 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130987.log.gz 2023-10-07T08:01:33.279 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140053.log 2023-10-07T08:01:33.279 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126756.log.gz 2023-10-07T08:01:33.279 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134957.log 2023-10-07T08:01:33.280 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135153.log.gz 2023-10-07T08:01:33.280 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.140053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135662.log 2023-10-07T08:01:33.280 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140053.log.gz 2023-10-07T08:01:33.280 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125839.log 2023-10-07T08:01:33.280 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134957.log.gz 2023-10-07T08:01:33.280 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136435.log 2023-10-07T08:01:33.281 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.135662.log.gz 2023-10-07T08:01:33.281 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125839.log: /var/log/ceph/ceph-client.admin.136435.log: gzip -5 --verbose -- 88.1%/var/log/ceph/ceph-client.admin.132947.log 2023-10-07T08:01:33.281 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.125839.log.gz 2023-10-07T08:01:33.281 INFO:teuthology.orchestra.run.smithi007.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.136435.log.gz 2023-10-07T08:01:33.281 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139759.log 2023-10-07T08:01:33.281 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132398.log 2023-10-07T08:01:33.281 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132947.log: /var/log/ceph/ceph-client.admin.139759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130472.log 2023-10-07T08:01:33.282 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139759.log.gz 2023-10-07T08:01:33.282 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132398.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.132947.log.gz 2023-10-07T08:01:33.282 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131232.log 2023-10-07T08:01:33.282 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132398.log.gz 2023-10-07T08:01:33.282 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138469.log 2023-10-07T08:01:33.282 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130472.log.gz 2023-10-07T08:01:33.283 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131232.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131232.log.gz 2023-10-07T08:01:33.283 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131618.log 2023-10-07T08:01:33.283 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138469.log.gz 2023-10-07T08:01:33.283 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125937.log 2023-10-07T08:01:33.283 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135006.log 2023-10-07T08:01:33.283 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131618.log.gz 2023-10-07T08:01:33.283 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135055.log 2023-10-07T08:01:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 88.3%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.125937.log.gz/var/log/ceph/ceph-client.admin.130374.log 2023-10-07T08:01:33.284 INFO:teuthology.orchestra.run.smithi007.stderr: 2023-10-07T08:01:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135006.log: /var/log/ceph/ceph-client.admin.135055.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.135006.log.gz 2023-10-07T08:01:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138224.log 2023-10-07T08:01:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130374.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135055.log.gz 2023-10-07T08:01:33.285 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130374.log.gz 2023-10-07T08:01:33.285 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135762.log 2023-10-07T08:01:33.285 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139207.log 2023-10-07T08:01:33.285 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138224.log: /var/log/ceph/ceph-client.admin.135762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135813.log 2023-10-07T08:01:33.285 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138224.log.gz 2023-10-07T08:01:33.285 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135762.log.gz 2023-10-07T08:01:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132504.log 2023-10-07T08:01:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135813.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139207.log.gz 2023-10-07T08:01:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134497.log 2023-10-07T08:01:33.286 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135813.log.gz 2023-10-07T08:01:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134858.log 2023-10-07T08:01:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126753.log 2023-10-07T08:01:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134497.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.132504.log.gz 2023-10-07T08:01:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134858.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134497.log.gz 2023-10-07T08:01:33.287 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134858.log.gz 2023-10-07T08:01:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126753.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.126753.log.gz 2023-10-07T08:01:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135251.log 2023-10-07T08:01:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138782.log 2023-10-07T08:01:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127382.log 2023-10-07T08:01:33.288 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135251.log.gz 2023-10-07T08:01:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126356.log 2023-10-07T08:01:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134399.log /var/log/ceph/ceph-client.admin.127382.log: 2023-10-07T08:01:33.288 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.138782.log.gz 2023-10-07T08:01:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126356.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127382.log.gz 2023-10-07T08:01:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134399.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126356.log.gz 2023-10-07T08:01:33.289 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134399.log.gz 2023-10-07T08:01:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135968.log 2023-10-07T08:01:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138175.log 2023-10-07T08:01:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126035.log 2023-10-07T08:01:33.289 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.135968.log.gz 2023-10-07T08:01:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139002.log 2023-10-07T08:01:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131569.log 2023-10-07T08:01:33.290 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138175.log.gz/var/log/ceph/ceph-client.admin.126035.log: 2023-10-07T08:01:33.290 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139002.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126035.log.gz 2023-10-07T08:01:33.290 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134448.log 2023-10-07T08:01:33.290 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131569.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.139002.log.gz 2023-10-07T08:01:33.290 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127173.log 2023-10-07T08:01:33.290 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131569.log.gz 2023-10-07T08:01:33.290 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132247.log 2023-10-07T08:01:33.291 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134448.log.gz 2023-10-07T08:01:33.291 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128117.log 2023-10-07T08:01:33.291 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127173.log.gz 2023-10-07T08:01:33.291 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135711.log 2023-10-07T08:01:33.291 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132247.log.gz 2023-10-07T08:01:33.291 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.128117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126990.log 2023-10-07T08:01:33.291 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135711.log: /var/log/ceph/ceph-client.admin.126990.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.135711.log.gz 2023-10-07T08:01:33.291 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.128117.log.gz 2023-10-07T08:01:33.292 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126990.log.gz 2023-10-07T08:01:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136166.log 2023-10-07T08:01:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139318.log 2023-10-07T08:01:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.136166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135327.log 2023-10-07T08:01:33.292 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.136166.log.gz 2023-10-07T08:01:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125584.log 2023-10-07T08:01:33.292 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.139318.log.gz 2023-10-07T08:01:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127578.log 2023-10-07T08:01:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125584.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-07T08:01:33.293 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135327.log.gz 2023-10-07T08:01:33.293 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.125584.log.gz 2023-10-07T08:01:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127578.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-07T08:01:33.294 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127578.log.gz 2023-10-07T08:01:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134054.log 2023-10-07T08:01:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.134054.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.134054.log.gz 2023-10-07T08:01:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126084.log 2023-10-07T08:01:33.295 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127970.log 2023-10-07T08:01:33.295 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126084.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126084.log.gz 2023-10-07T08:01:33.295 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127970.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127970.log.gz 2023-10-07T08:01:33.295 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137954.log 2023-10-07T08:01:33.295 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132849.log 2023-10-07T08:01:33.296 INFO:teuthology.orchestra.run.smithi172.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-07T08:01:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.137954.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137954.log.gz 2023-10-07T08:01:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132849.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132849.log.gz 2023-10-07T08:01:33.299 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137067.log 2023-10-07T08:01:33.300 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133382.log 2023-10-07T08:01:33.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.137067.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137067.log.gz 2023-10-07T08:01:33.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.133382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128861.log 2023-10-07T08:01:33.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.128861.log: 90.8% -- replaced with /var/log/ceph/ceph-client.0.128861.log.gz 2023-10-07T08:01:33.300 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127431.log 2023-10-07T08:01:33.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127431.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127431.log.gz 2023-10-07T08:01:33.300 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127725.log 2023-10-07T08:01:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127725.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127725.log.gz 2023-10-07T08:01:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128068.log 2023-10-07T08:01:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.128068.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128068.log.gz 2023-10-07T08:01:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-07T08:01:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126746.log 2023-10-07T08:01:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126746.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126746.log.gz 2023-10-07T08:01:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133978.log 2023-10-07T08:01:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.133978.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133978.log.gz 2023-10-07T08:01:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140257.log 2023-10-07T08:01:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.140257.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.140257.log.gz 2023-10-07T08:01:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.113154.log 2023-10-07T08:01:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.113154.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.113154.log.gz 2023-10-07T08:01:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132455.log 2023-10-07T08:01:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132455.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.132455.log.gz 2023-10-07T08:01:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132149.log 2023-10-07T08:01:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132149.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132149.log.gz 2023-10-07T08:01:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136313.log 2023-10-07T08:01:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.136313.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136313.log.gz 2023-10-07T08:01:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125641.log 2023-10-07T08:01:33.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125641.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125641.log.gz 2023-10-07T08:01:33.304 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137525.log 2023-10-07T08:01:33.304 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.137525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127921.log 2023-10-07T08:01:33.309 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127921.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127921.log.gz 2023-10-07T08:01:33.309 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.129592.log 2023-10-07T08:01:33.322 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.2.129592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139465.log 2023-10-07T08:01:33.322 INFO:teuthology.orchestra.run.smithi007.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.2.129592.log.gz 2023-10-07T08:01:33.333 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139416.log 2023-10-07T08:01:33.339 INFO:teuthology.orchestra.run.smithi172.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-07T08:01:33.344 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.139465.log.gz 2023-10-07T08:01:33.346 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125433.log 2023-10-07T08:01:33.346 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139416.log.gz 2023-10-07T08:01:33.347 INFO:teuthology.orchestra.run.smithi007.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.133382.log.gz 2023-10-07T08:01:33.349 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125433.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125433.log.gz 2023-10-07T08:01:33.350 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-07T08:01:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: 83.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-07T08:01:33.351 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132653.log 2023-10-07T08:01:33.353 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132653.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132653.log.gz 2023-10-07T08:01:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138518.log 2023-10-07T08:01:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138518.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138518.log.gz 2023-10-07T08:01:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136362.log 2023-10-07T08:01:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.136362.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.136362.log.gz 2023-10-07T08:01:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139051.log 2023-10-07T08:01:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139051.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139051.log.gz 2023-10-07T08:01:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140004.log 2023-10-07T08:01:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.140004.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.140004.log.gz 2023-10-07T08:01:33.358 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134742.log 2023-10-07T08:01:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134742.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134742.log.gz 2023-10-07T08:01:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125327.log 2023-10-07T08:01:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125327.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125327.log.gz 2023-10-07T08:01:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129742.log 2023-10-07T08:01:33.362 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.129742.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129742.log.gz 2023-10-07T08:01:33.362 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134908.log 2023-10-07T08:01:33.363 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134908.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134908.log.gz 2023-10-07T08:01:33.363 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139367.log 2023-10-07T08:01:33.364 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139367.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.139367.log.gz 2023-10-07T08:01:33.364 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140200.log 2023-10-07T08:01:33.365 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.140200.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140200.log.gz 2023-10-07T08:01:33.365 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130668.log 2023-10-07T08:01:33.367 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130668.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130668.log.gz 2023-10-07T08:01:33.367 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139808.log 2023-10-07T08:01:33.368 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139808.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139808.log.gz 2023-10-07T08:01:33.368 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132898.log 2023-10-07T08:01:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132898.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.132898.log.gz 2023-10-07T08:01:33.369 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127091.log 2023-10-07T08:01:33.370 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127091.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127091.log.gz 2023-10-07T08:01:33.370 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127529.log 2023-10-07T08:01:33.371 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127529.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127529.log.gz 2023-10-07T08:01:33.372 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131330.log 2023-10-07T08:01:33.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131330.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.131330.log.gz 2023-10-07T08:01:33.373 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126747.log 2023-10-07T08:01:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126747.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126747.log.gz 2023-10-07T08:01:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140331.log 2023-10-07T08:01:33.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.140331.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140331.log.gz 2023-10-07T08:01:33.375 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131953.log 2023-10-07T08:01:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131953.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.131953.log.gz 2023-10-07T08:01:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131904.log 2023-10-07T08:01:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131904.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.131904.log.gz 2023-10-07T08:01:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135870.log 2023-10-07T08:01:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135870.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135870.log.gz 2023-10-07T08:01:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139906.log 2023-10-07T08:01:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.139906.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139906.log.gz 2023-10-07T08:01:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127872.log 2023-10-07T08:01:33.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127872.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127872.log.gz 2023-10-07T08:01:33.381 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127480.log 2023-10-07T08:01:33.382 INFO:teuthology.orchestra.run.smithi172.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-07T08:01:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127480.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127480.log.gz 2023-10-07T08:01:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138420.log 2023-10-07T08:01:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138420.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138420.log.gz 2023-10-07T08:01:33.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135202.log 2023-10-07T08:01:33.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.135202.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135202.log.gz 2023-10-07T08:01:33.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132002.log 2023-10-07T08:01:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132002.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.132002.log.gz 2023-10-07T08:01:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131085.log 2023-10-07T08:01:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131085.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131085.log.gz 2023-10-07T08:01:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126348.log 2023-10-07T08:01:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126348.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126348.log.gz 2023-10-07T08:01:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130717.log 2023-10-07T08:01:33.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130717.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.130717.log.gz 2023-10-07T08:01:33.390 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132347.log 2023-10-07T08:01:33.391 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132347.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.132347.log.gz 2023-10-07T08:01:33.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130766.log 2023-10-07T08:01:33.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.130766.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130766.log.gz 2023-10-07T08:01:33.392 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137856.log 2023-10-07T08:01:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.137856.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137856.log.gz 2023-10-07T08:01:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124985.log 2023-10-07T08:01:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.124985.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.124985.log.gz 2023-10-07T08:01:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127169.log 2023-10-07T08:01:33.395 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127169.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.127169.log.gz 2023-10-07T08:01:33.395 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134350.log 2023-10-07T08:01:33.397 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134350.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.134350.log.gz 2023-10-07T08:01:33.397 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.136798.log 2023-10-07T08:01:33.398 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.136798.log: 88.4% -- replaced with /var/log/ceph/ceph-client.0.136798.log.gz 2023-10-07T08:01:33.399 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138880.log 2023-10-07T08:01:33.399 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138880.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138880.log.gz 2023-10-07T08:01:33.399 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127093.log 2023-10-07T08:01:33.400 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127093.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127093.log.gz 2023-10-07T08:01:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132751.log 2023-10-07T08:01:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132751.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.132751.log.gz 2023-10-07T08:01:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125739.log 2023-10-07T08:01:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.125739.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125739.log.gz 2023-10-07T08:01:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126754.log 2023-10-07T08:01:33.404 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126754.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126754.log.gz 2023-10-07T08:01:33.404 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-07T08:01:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137164.log 2023-10-07T08:01:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.137164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131452.log 2023-10-07T08:01:33.427 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137164.log.gz 2023-10-07T08:01:33.448 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132553.log 2023-10-07T08:01:33.449 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131452.log.gz 2023-10-07T08:01:33.453 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132553.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.132553.log.gz 2023-10-07T08:01:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132100.log 2023-10-07T08:01:33.475 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.132100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127092.log 2023-10-07T08:01:33.476 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132100.log.gz 2023-10-07T08:01:33.484 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.127092.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127092.log.gz 2023-10-07T08:01:33.484 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126354.log 2023-10-07T08:01:33.495 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.126354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134693.log 2023-10-07T08:01:33.496 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.126354.log.gz 2023-10-07T08:01:33.501 INFO:teuthology.orchestra.run.smithi007.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-07T08:01:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134693.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134693.log.gz 2023-10-07T08:01:33.503 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131183.log 2023-10-07T08:01:33.504 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131183.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131183.log.gz 2023-10-07T08:01:33.504 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134130.log 2023-10-07T08:01:33.505 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.134130.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.134130.log.gz 2023-10-07T08:01:33.505 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140102.log 2023-10-07T08:01:33.506 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.140102.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140102.log.gz 2023-10-07T08:01:33.507 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138683.log 2023-10-07T08:01:33.510 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.138683.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.138683.log.gz 2023-10-07T08:01:33.510 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131855.log 2023-10-07T08:01:33.511 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.131855.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131855.log.gz 2023-10-07T08:01:33.612 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-07T08:01:33.639 INFO:teuthology.orchestra.run.smithi172.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-07T08:01:33.718 INFO:teuthology.orchestra.run.smithi172.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-07T08:01:33.853 INFO:teuthology.orchestra.run.smithi172.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-07T08:01:33.898 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-07T08:01:33.942 INFO:teuthology.orchestra.run.smithi007.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-07T08:01:33.944 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-07T08:01:33.953 INFO:teuthology.orchestra.run.smithi172.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-07T08:01:33.955 INFO:teuthology.orchestra.run.smithi172.stderr: 2023-10-07T08:01:33.955 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m0.755s 2023-10-07T08:01:33.955 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m2.570s 2023-10-07T08:01:33.955 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m0.120s 2023-10-07T08:01:34.034 INFO:teuthology.orchestra.run.smithi007.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-07T08:01:35.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-10-07T08:01:35.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-10-07T08:01:35.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-10-07T08:01:35.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-10-07T08:01:35.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-10-07T08:01:35.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-10-07T08:01:35.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-10-07T08:01:35.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-10-07T08:01:37.438 INFO:teuthology.orchestra.run.smithi007.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.137525.log.gz 2023-10-07T08:01:42.787 INFO:teuthology.orchestra.run.smithi007.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-07T08:01:42.789 INFO:teuthology.orchestra.run.smithi007.stderr: 2023-10-07T08:01:42.790 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m9.611s 2023-10-07T08:01:42.790 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m16.615s 2023-10-07T08:01:42.790 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.758s 2023-10-07T08:01:42.790 INFO:tasks.ceph:Archiving logs... 2023-10-07T08:01:42.790 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/remote/smithi007/log 2023-10-07T08:01:42.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-07T08:01:45.211 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/remote/smithi172/log 2023-10-07T08:01:45.212 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-07T08:01:45.671 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-07T08:01:45.684 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-07T08:01:45.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-07T08:01:45.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-07T08:01:45.780 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-07T08:01:45.790 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-07T08:01:45.790 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T08:01:45.793 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T08:01:45.812 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2023-10-07T08:01:45.813 INFO:teuthology.orchestra.run.smithi172.stderr:bash: ntpq: command not found 2023-10-07T08:01:45.819 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T08:01:45.819 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2023-10-07T08:01:45.819 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 -192us[ -163us] +/- 80ms 2023-10-07T08:01:45.819 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -191us[ -161us] +/- 76ms 2023-10-07T08:01:45.820 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +318us[ +348us] +/- 60ms 2023-10-07T08:01:45.820 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T08:01:45.820 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T08:01:45.820 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2023-10-07T08:01:45.820 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 -188us[ -161us] +/- 80ms 2023-10-07T08:01:45.820 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -188us[ -161us] +/- 76ms 2023-10-07T08:01:45.821 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +329us[ +356us] +/- 60ms 2023-10-07T08:01:45.821 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T08:01:45.821 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-07T08:01:45.831 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-07T08:01:45.832 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-07T08:01:45.841 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-07T08:01:45.976 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-07T08:01:46.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T08:01:46.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T08:01:46.114 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T08:01:46.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T08:01:46.148 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-07T08:01:46.158 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-07T08:01:46.169 INFO:teuthology.task.internal:Duration was 933.340036 seconds 2023-10-07T08:01:46.169 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-07T08:01:46.179 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-07T08:01:46.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-07T08:01:46.182 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-07T08:01:46.236 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T08:01:46.241 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T08:01:46.573 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-07T08:01:46.573 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2023-10-07T08:01:46.573 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-07T08:01:46.606 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2023-10-07T08:01:46.607 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-07T08:01:46.630 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-07T08:01:46.630 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-07T08:01:46.652 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-07T08:01:46.710 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-07T08:01:46.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-07T08:01:46.714 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-07T08:01:46.821 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-07T08:01:46.836 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-07T08:01:46.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-07T08:01:46.872 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-07T08:01:46.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-07T08:01:46.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-07T08:01:46.920 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-07T08:01:46.952 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2023-10-07T08:01:46.968 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2023-10-07T08:01:46.981 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-07T08:01:47.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T08:01:47.032 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-07T08:01:47.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T08:01:47.050 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-07T08:01:47.061 INFO:teuthology.task.internal:Transferring archived files... 2023-10-07T08:01:47.061 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/remote/smithi007 2023-10-07T08:01:47.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-07T08:01:47.119 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223/remote/smithi172 2023-10-07T08:01:47.119 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-07T08:01:47.157 INFO:teuthology.task.internal:Removing archive directory... 2023-10-07T08:01:47.158 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-07T08:01:47.160 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-07T08:01:47.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-07T08:01:47.220 INFO:teuthology.task.internal:Not uploading archives. 2023-10-07T08:01:47.220 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-07T08:01:47.232 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-07T08:01:47.233 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-07T08:01:47.237 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-07T08:01:47.256 INFO:teuthology.orchestra.run.smithi007.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 7 08:01 /home/ubuntu/cephtest 2023-10-07T08:01:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: 393225 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 7 08:01 /home/ubuntu/cephtest 2023-10-07T08:01:47.268 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-07T08:01:47.280 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-07T08:01:47.309 INFO:teuthology.nuke:Checking targets against current locks 2023-10-07T08:01:47.335 DEBUG:teuthology.nuke:shortname: smithi007 2023-10-07T08:01:47.335 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T08:01:47.363 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 07:37:17.738859', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDvzTKOQgOGweamAbVUNfrC8pj/dSyjZ9V0tg0+C7IOj4n90Q1to5OmKeP+Jy9J1prx3MYhGj8ZNLJlE75U6p9w='} 2023-10-07T08:01:47.368 DEBUG:teuthology.nuke:shortname: smithi172 2023-10-07T08:01:47.368 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T08:01:47.391 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_00:40:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7416223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 07:37:17.736779', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD/VRA/QSRFLcfZ0A2n4Zd6jOOh9njj/CIoNcl1+A3795u1aCYQmZKDV32dGZGcnYvugk/tG7M6tUZ9JXtW67cA='} 2023-10-07T08:01:47.422 INFO:teuthology.orchestra.console.smithi007:Power off 2023-10-07T08:01:47.423 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-07T08:01:47.444 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2023-10-07T08:01:47.445 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T08:01:47.461 INFO:teuthology.orchestra.console.smithi172:Power off 2023-10-07T08:01:47.462 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-07T08:01:47.473 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2023-10-07T08:01:47.485 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2023-10-07T08:01:47.486 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T08:01:47.512 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-10-07T08:01:51.474 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T08:01:51.513 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T08:01:51.588 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2023-10-07T08:01:51.623 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-10-07T08:01:55.590 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T08:01:55.624 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T08:01:55.701 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2023-10-07T08:01:55.735 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2023-10-07T08:01:55.802 INFO:teuthology.orchestra.console.smithi007:Power off completed 2023-10-07T08:01:55.836 INFO:teuthology.orchestra.console.smithi172:Power off completed 2023-10-07T08:01:55.940 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} duration: 933.3400356769562 failure_reason: 'Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=605d114cac0b40c484d7c055708d070a status: fail success: false 2023-10-07T08:01:55.941 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-07T08:01:56.020 INFO:teuthology.run:FAIL